[GitHub] spark issue #21564: [SPARK-24556][SQL] ReusedExchange should rewrite output ...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21564 **[Test build #91856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91856/testReport)** for PR 21564 at commit

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195542619 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195542491 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -304,6 +305,11 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195542018 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -98,14 +101,53 @@ class ExecutorSummary private[spark]( val

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195541136 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -234,8 +272,18 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #21564: [SPARK-24556][SQL] ReusedExchange should rewrite output ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21564 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21564: [SPARK-24556][SQL] ReusedExchange should rewrite output ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21564 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91855/ Test PASSed. ---

[GitHub] spark issue #21564: [SPARK-24556][SQL] ReusedExchange should rewrite output ...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21564 **[Test build #91855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91855/testReport)** for PR 21564 at commit

[GitHub] spark issue #21569: [SPARK-24563][PYTHON]Catch TypeError when testing existe...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/152/

[GitHub] spark issue #21569: [SPARK-24563][PYTHON]Catch TypeError when testing existe...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21569 **[Test build #91865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91865/testReport)** for PR 21569 at commit

[GitHub] spark issue #21569: [SPARK-24563][PYTHON]Catch TypeError when testing existe...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21569 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21569: [SPARK-24563][PYTHON]Catch TypeError when testing existe...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4043/

[GitHub] spark issue #21569: [SPARK-24563][PYTHON]Catch TypeError when testing existe...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21569 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195539837 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -169,6 +182,31 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195538848 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -169,6 +182,31 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #21569: Catch TypeError when testing existence of HiveCon...

2018-06-14 Thread icexelloss
GitHub user icexelloss opened a pull request: https://github.com/apache/spark/pull/21569 Catch TypeError when testing existence of HiveConf when creating pysp… …ark shell ## What changes were proposed in this pull request? This PR catches TypeError when testing

[GitHub] spark issue #21527: [SPARK-24519] MapStatus has 2000 hardcoded

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91850/ Test PASSed. ---

[GitHub] spark issue #21527: [SPARK-24519] MapStatus has 2000 hardcoded

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21527: [SPARK-24519] MapStatus has 2000 hardcoded

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21527 **[Test build #91850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91850/testReport)** for PR 21527 at commit

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195535296 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark issue #21559: [SPARK-24525][SS] Provide an option to limit number of r...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21559 **[Test build #91864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91864/testReport)** for PR 21559 at commit

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195534314 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -93,6 +96,9 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-06-14 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r195533972 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1751,7 +1753,7 @@ class DAGScheduler(

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20636 **[Test build #91863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91863/testReport)** for PR 20636 at commit

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91863/ Test FAILed. ---

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-14 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/21366 If last round's comments are addressed, LGTM from me. Important behavior to check is - the snapshot, and creating replacement executors based on captured snapshot. ---

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20636 **[Test build #91863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91863/testReport)** for PR 20636 at commit

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/151/

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4042/

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-14 Thread dvogelbacher
Github user dvogelbacher commented on the issue: https://github.com/apache/spark/pull/21366 Agree with @mccheah on not blocking this on a design doc. This PR strictly improves the management of executor states in k8s compared to how it was done before. So we really should get this

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-14 Thread Silberlocke
Github user Silberlocke commented on the issue: https://github.com/apache/spark/pull/21366 Agree with @mccheah on not blocking this on a design doc. This PR strictly improves the management of executor states in k8s compared to how it was done before. So we really should get this

[GitHub] spark issue #21441: [DO-NOT-MERGE] Run tests against hadoop-3.1 to see the t...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21441 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21441: [DO-NOT-MERGE] Run tests against hadoop-3.1 to see the t...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21441 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91851/ Test FAILed. ---

[GitHub] spark issue #21441: [DO-NOT-MERGE] Run tests against hadoop-3.1 to see the t...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21441 **[Test build #91851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91851/testReport)** for PR 21441 at commit

[GitHub] spark pull request #21559: [SPARK-24525][SS] Provide an option to limit numb...

2018-06-14 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21559#discussion_r195516859 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/memoryV2.scala --- @@ -110,40 +126,61 @@ class MemorySinkV2 extends

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91846/ Test PASSed. ---

[GitHub] spark pull request #21559: [SPARK-24525][SS] Provide an option to limit numb...

2018-06-14 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21559#discussion_r195516533 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala --- @@ -294,6 +333,16 @@ class MemorySink(val schema:

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #91846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91846/testReport)** for PR 21221 at commit

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195513995 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -56,17 +58,44 @@

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/150/

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4041/

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-14 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21366 > @mccheah could you add a design doc for future reference and so that new contributors can understand better the rationale behind this. There is some description in the JIRA ticket but not enough

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195512808 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195512430 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,141 @@

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21546 cc @icexelloss --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21546 Thanks @viirya and @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195512219 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -56,17 +58,44 @@

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195512218 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala --- @@ -1318,18 +1318,52 @@ class

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21546 **[Test build #91862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91862/testReport)** for PR 21546 at commit

[GitHub] spark issue #21559: [SPARK-24525][SS] Provide an option to limit number of r...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21559 **[Test build #91861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91861/testReport)** for PR 21559 at commit

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-14 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21515 FYI I'm playing with building 2.1.3 using the docker-based approach, and there are a few things that I need to adjust before that works. I hope to update this PR later today. ---

[GitHub] spark issue #21531: [SPARK-24521][SQL][TEST] Fix ineffective test in CachedT...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91848/ Test PASSed. ---

[GitHub] spark issue #21531: [SPARK-24521][SQL][TEST] Fix ineffective test in CachedT...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21531 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21531: [SPARK-24521][SQL][TEST] Fix ineffective test in CachedT...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21531 **[Test build #91848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91848/testReport)** for PR 21531 at commit

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195504451 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/python/PythonSQLUtils.scala --- @@ -34,17 +34,36 @@ private[sql] object PythonSQLUtils {

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #91860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91860/testReport)** for PR 21068 at commit

[GitHub] spark issue #21568: [SPARK-24562][TESTS] Support different configs for same ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21568 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195502588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/python/PythonSQLUtils.scala --- @@ -34,17 +34,36 @@ private[sql] object PythonSQLUtils {

[GitHub] spark issue #21568: [SPARK-24562][TESTS] Support different configs for same ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4040/

[GitHub] spark issue #21568: [SPARK-24562][TESTS] Support different configs for same ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21568 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21546 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21568: [SPARK-24562][TESTS] Support different configs for same ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/149/

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195501939 --- Diff: python/pyspark/serializers.py --- @@ -184,24 +184,28 @@ def loads(self, obj): raise NotImplementedError -class

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195501843 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala --- @@ -1318,18 +1318,52 @@ class

[GitHub] spark issue #21529: [SPARK-24495][SQL] EnsureRequirement returns wrong plan ...

2018-06-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21529 Adding new queries to `SQLQueryTestSuite` is the best way to do it in the current infrastructure. Do your best to cover all the join algorithms for different input data and join types? ---

[GitHub] spark issue #21568: [SPARK-24562][TESTS] Support different configs for same ...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21568 **[Test build #91859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91859/testReport)** for PR 21568 at commit

[GitHub] spark issue #21568: [SPARK-24562][TESTS] Support different configs for same ...

2018-06-14 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21568 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21568: [SPARK-24562][TESTS] Support different configs fo...

2018-06-14 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/21568 [SPARK-24562][TESTS] Support different configs for same test in SQLQueryTestSuite ## What changes were proposed in this pull request? The PR proposes to add support for running the same

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195499089 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/python/PythonSQLUtils.scala --- @@ -34,17 +34,36 @@ private[sql] object PythonSQLUtils {

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195498764 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3236,13 +3236,49 @@ class Dataset[T] private[sql]( }

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91852/ Test FAILed. ---

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20611 **[Test build #91852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91852/testReport)** for PR 20611 at commit

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-14 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r195497043 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3236,13 +3236,49 @@ class Dataset[T] private[sql]( }

[GitHub] spark issue #21550: [SPARK-24543][SQL] Support any type as DDL string for fr...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21550 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91844/ Test PASSed. ---

[GitHub] spark issue #21550: [SPARK-24543][SQL] Support any type as DDL string for fr...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21550 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21529: [SPARK-24495][SQL] EnsureRequirement returns wrong plan ...

2018-06-14 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21529 Thanks @gatorsmile. Sorry, may I ask what you think about https://github.com/apache/spark/pull/21529#issuecomment-396707622? Thanks. ---

[GitHub] spark issue #21550: [SPARK-24543][SQL] Support any type as DDL string for fr...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21550 **[Test build #91844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91844/testReport)** for PR 21550 at commit

[GitHub] spark issue #21558: [SPARK-24552][SQL] Use task ID instead of attempt number...

2018-06-14 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21558 sorry trying to catch up on this thread. Are we saying this is a bug in the existing output committer as well when we have a fetch failure? ---

[GitHub] spark issue #21476: [SPARK-24446][yarn] Properly quote library path for YARN...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21476 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21476: [SPARK-24446][yarn] Properly quote library path for YARN...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21476 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4039/

[GitHub] spark pull request #21529: [SPARK-24495][SQL] EnsureRequirement returns wron...

2018-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21529 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21529: [SPARK-24495][SQL] EnsureRequirement returns wrong plan ...

2018-06-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21529 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20636 **[Test build #91858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91858/testReport)** for PR 20636 at commit

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91858/ Test FAILed. ---

[GitHub] spark issue #21529: [SPARK-24495][SQL] EnsureRequirement returns wrong plan ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21529 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21529: [SPARK-24495][SQL] EnsureRequirement returns wrong plan ...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21529 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91843/ Test PASSed. ---

[GitHub] spark issue #21529: [SPARK-24495][SQL] EnsureRequirement returns wrong plan ...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21529 **[Test build #91843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91843/testReport)** for PR 21529 at commit

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20636 **[Test build #91858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91858/testReport)** for PR 20636 at commit

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21288 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4037/

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20636 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4038/

<    1   2   3   4   5   6   7   8   >