[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21308 **[Test build #94814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94814/testReport)** for PR 21308 at commit [`db77b9a`](https://github.com/apache/spark/commit/db

[GitHub] spark pull request #21308: SPARK-24253: Add DeleteSupport mix-in for DataSou...

2018-08-15 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21308#discussion_r210382412 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/DeleteSupport.java --- @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21308 @rxin, I've updated this API to use `Filter` instead of `Expression`. I'd ideally like to get it in soon if you guys have a chance to review it. It's pretty small. cc @cloud-fan ---

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21308 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2219/

[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21308 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2220/

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #94815 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94815/testReport)** for PR 22106 at commit [`621099e`](https://github.com/apache/spark/commit/62

[GitHub] spark pull request #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak an...

2018-08-15 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22106#discussion_r210383608 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaContinuousReader.scala --- @@ -216,7 +216,7 @@ class KafkaContinuousInp

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2221/

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #94816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94816/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/63

[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21308 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified//

[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21308 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22013 **[Test build #94817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94817/testReport)** for PR 22013 at commit [`58b60b2`](https://github.com/apache/spark/commit/58

[GitHub] spark issue #21308: SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21308 **[Test build #94818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94818/testReport)** for PR 21308 at commit [`e32e6c4`](https://github.com/apache/spark/commit/e3

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4259 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4259/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4261 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4261/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4260 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4260/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4262 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4262/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak an...

2018-08-15 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22106#discussion_r210387003 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaContinuousSinkSuite.scala --- @@ -40,12 +40,7 @@ class KafkaContinuousS

[GitHub] spark issue #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22013 **[Test build #94819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94819/testReport)** for PR 22013 at commit [`2f4943f`](https://github.com/apache/spark/commit/2f

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #94820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94820/testReport)** for PR 22104 at commit [`38f3dbb`](https://github.com/apache/spark/commit/38

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21889 **[Test build #94805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94805/testReport)** for PR 21889 at commit [`8d822ee`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94804/testReport)** for PR 21977 at commit [`6810fc7`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21889 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94805/ Test PASSed. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94820/ Test FAILed. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21889 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94804/ Test FAILed. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #94820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94820/testReport)** for PR 22104 at commit [`38f3dbb`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r210390399 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala --- @@ -117,15 +117,16 @@ abstract class EvalPythonExec(udf

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2223/

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() da...

2018-08-15 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/22112 [WIP][SPARK-23243][Core] Fix RDD.repartition() data correctness issue ## What changes were proposed in this pull request? An alternative fix for https://github.com/apache/spark/pull/21698

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94821/testReport)** for PR 22112 at commit [`1f9f6e5`](https://github.com/apache/spark/commit/1f

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r210390770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -133,6 +134,9 @@ object ExtractPythonUDFs extends

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21698 I tried a prototype to fix the handling of fetch failure, seems not that hard: https://github.com/apache/spark/pull/22112 ---

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r210391237 --- Diff: python/pyspark/sql/tests.py --- @@ -3367,6 +3367,35 @@ def test_ignore_column_of_all_nulls(self): finally: shutil.

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94821/ Test FAILed. ---

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94821 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94821/testReport)** for PR 22112 at commit [`1f9f6e5`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2224/

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2225/

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22009 **[Test build #94823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94823/testReport)** for PR 22009 at commit [`2018981`](https://github.com/apache/spark/commit/20

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #94822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94822/testReport)** for PR 22104 at commit [`fa7a869`](https://github.com/apache/spark/commit/fa

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2226/

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22109: [SPARK-25120][CORE][HistoryServer]Fix the problem of Eve...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22109 **[Test build #94807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94807/testReport)** for PR 22109 at commit [`26ca9c2`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22109: [SPARK-25120][CORE][HistoryServer]Fix the problem of Eve...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22109 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94807/ Test FAILed. ---

[GitHub] spark issue #22109: [SPARK-25120][CORE][HistoryServer]Fix the problem of Eve...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22109 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #94815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94815/testReport)** for PR 22106 at commit [`621099e`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94815/ Test PASSed. ---

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22112 haven't look at code yet, does it just fail with ResultTask then? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apach

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4262/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #94816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94816/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94816/ Test PASSed. ---

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22110: [SPARK-25122][SQL] Deduplication of supports equals code

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22110 **[Test build #94824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94824/testReport)** for PR 22110 at commit [`d0b0552`](https://github.com/apache/spark/commit/d0

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4259/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4260/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4261/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...

2018-08-15 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22001 LGTM. Merged into master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-15 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22085 @HyukjinKwon Thanks for the feedback! We will replace the py4j route by a special implementation that can only trigger "context.barrier()" in JVM. --- --

[GitHub] spark pull request #22113: [SPARK-25126] Lazily create Reader for orc files

2018-08-15 Thread raofu
GitHub user raofu opened a pull request: https://github.com/apache/spark/pull/22113 [SPARK-25126] Lazily create Reader for orc files ## What changes were proposed in this pull request? Currently Reader is created for every orc file under the directory and then the first one

[GitHub] spark pull request #22001: [SPARK-24819][CORE] Fail fast when no enough slot...

2018-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22001 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22113: [SPARK-25126] Lazily create Reader for orc files

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22113 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22113: [SPARK-25126] Lazily create Reader for orc files

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22113 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22113: [SPARK-25126] Lazily create Reader for orc files

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22113 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22045: [SPARK-23940][SQL] Add transform_values SQL funct...

2018-08-15 Thread codeatri
Github user codeatri commented on a diff in the pull request: https://github.com/apache/spark/pull/22045#discussion_r210402976 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -497,6 +497,60 @@ case class ArrayAggreg

[GitHub] spark issue #21339: [SPARK-24287][Core] Spark -packages option should suppor...

2018-08-15 Thread fangshil
Github user fangshil commented on the issue: https://github.com/apache/spark/pull/21339 thanks for the follow up. i will rebase this patch to latest master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apac

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21537 I am fine to not revert it since it is too late. So many related PRs have been merged, but we need to seriously consider writing and reviewing the design docs before changing the code generation

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2227/

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21306 **[Test build #94825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94825/testReport)** for PR 21306 at commit [`622180a`](https://github.com/apache/spark/commit/62

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-08-15 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21306 @rxin, I've updated this to use a new interface, `PartitionTransform`, instead of `Expression`. This is used to pass well-known transformations when creating tables, like `Filter` is used to pass a s

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r210410738 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala --- @@ -117,15 +117,16 @@ abstract class EvalPythonExec(udf

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22112 nevermind see you have an abortStage in there for ResultTask --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #94826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94826/testReport)** for PR 22104 at commit [`8409611`](https://github.com/apache/spark/commit/84

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r210414941 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala --- @@ -117,15 +117,18 @@ abstract class EvalPythonExec(udf

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #94826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94826/testReport)** for PR 22104 at commit [`8409611`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94826/ Test FAILed. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2228/

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22013 **[Test build #94811 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94811/testReport)** for PR 22013 at commit [`fb885f4`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-15 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22112 need to look at in more detail but if its straight forward could at least do this short term for the repartition case. I guess I question whether we really want to do it for zip and other

[GitHub] spark issue #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22013 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94811/ Test FAILed. ---

[GitHub] spark issue #22109: [SPARK-25120][CORE][HistoryServer]Fix the problem of Eve...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22109 **[Test build #4263 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4263/testReport)** for PR 22109 at commit [`26ca9c2`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22013 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21661: [SPARK-24685][build] Restore support for building old Ha...

2018-08-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21661 Even though there's talk of EOLing 2.1, I think this can still be helpful later since there's also talk of removing the hadoop-2.6 build from 2.4, and the code here makes it easy to make those adjust

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4264 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4264/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4265 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4265/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4266 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4266/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4267 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4267/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22106: [SPARK-25116][TESTS]Fix the Kafka cluster leak and clean...

2018-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22106 **[Test build #4268 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4268/testReport)** for PR 22106 at commit [`63cc11d`](https://github.com/apache/spark/commit/6

<    1   2   3   4   5   6   >