[GitHub] spark issue #22186: [SPARK-25183][SQL] Spark HiveServer2 to use Spark Shutdo...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22186 **[Test build #95114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95114/testReport)** for PR 22186 at commit

[GitHub] spark issue #22176: [SPARK-25181][CORE] Limit Thread Pool size in BlockManag...

2018-08-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22176 LGTM. Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22042 **[Test build #95115 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95115/testReport)** for PR 22042 at commit

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22042 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22042 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22186: [SPARK-25183][SQL] Spark HiveServer2 to use Spark Shutdo...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22186 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22186: [SPARK-25183][SQL] Spark HiveServer2 to use Spark Shutdo...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22186 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95113/ Test PASSed. ---

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #95113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95113/testReport)** for PR 22121 at commit

[GitHub] spark issue #22186: [SPARK-25183][SQL] Spark HiveServer2 to use Spark Shutdo...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22186 **[Test build #95114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95114/testReport)** for PR 22186 at commit

[GitHub] spark pull request #22186: [SPARK-25183][SQL] Spark HiveServer2 to use Spark...

2018-08-22 Thread steveloughran
GitHub user steveloughran opened a pull request: https://github.com/apache/spark/pull/22186 [SPARK-25183][SQL] Spark HiveServer2 to use Spark ShutdownHookManager ## What changes were proposed in this pull request? Switch `org.apache.hive.service.server.HiveServer2` to

[GitHub] spark issue #22176: [SPARK-25181][CORE] Limit Thread Pool size in BlockManag...

2018-08-22 Thread mukulmurthy
Github user mukulmurthy commented on the issue: https://github.com/apache/spark/pull/22176 @zsxwing , @JoshRosen , @marmbrus for review and merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95112/ Test PASSed. ---

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2451/ ---

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 **[Test build #95112 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95112/testReport)** for PR 22146 at commit

[GitHub] spark issue #22100: [SPARK-25105][PySpark] [SQL]Include PandasUDFType in the...

2018-08-22 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22100 merged to master, thanks @kevinyu98 ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22100: [SPARK-25105][PySpark] [SQL]Include PandasUDFType...

2018-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22100 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2451/ ---

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #95113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95113/testReport)** for PR 22121 at commit

[GitHub] spark issue #20838: [SPARK-23698][Python] Resolve undefined names in Python ...

2018-08-22 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20838 merged to master, thanks @cclauss ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20838: [SPARK-23698][Python] Resolve undefined names in ...

2018-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20838 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95104/ Test FAILed. ---

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #95104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95104/testReport)** for PR 20345 at commit

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 **[Test build #95112 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95112/testReport)** for PR 22146 at commit

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22146 @yifeih can we also modify `docs/running-on-kubernetes.md` and have a section specific to using this feature? Mark it as experimental also. ---

[GitHub] spark pull request #22042: [SPARK-25005][SS]Support non-consecutive offsets ...

2018-08-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22042#discussion_r212033844 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala --- @@ -337,6 +338,7 @@ private[kafka010] case

[GitHub] spark issue #22157: [SPARK-25126] Avoid creating Reader for all orc files

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22157 **[Test build #4285 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4285/testReport)** for PR 22157 at commit

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22146 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22042: [SPARK-25005][SS]Support non-consecutive offsets ...

2018-08-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22042#discussion_r212032759 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala --- @@ -597,6 +614,254 @@ abstract class

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r212032223 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- + +*

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r212031015 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark issue #22171: [SPARK-25177][SQL] When dataframe decimal type column ha...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22171 **[Test build #95111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95111/testReport)** for PR 22171 at commit

[GitHub] spark issue #22171: [SPARK-25177][SQL] When dataframe decimal type column ha...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22171 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22171: [SPARK-25177][SQL] When dataframe decimal type column ha...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22171 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22171: [SPARK-25177][SQL] When dataframe decimal type column ha...

2018-08-22 Thread vinodkc
Github user vinodkc commented on the issue: https://github.com/apache/spark/pull/22171 @viirya, This issue is not only related to Dataset.show but also in dataset write operations. External databases like netezza fails save the result due to scientific notation on "0" values .

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r212027641 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark issue #22157: [SPARK-25126] Avoid creating Reader for all orc files

2018-08-22 Thread raofu
Github user raofu commented on the issue: https://github.com/apache/spark/pull/22157 I fixed the test by making the first file the corrupted file. @srowen, can you help kick off a Jenkins run? --- - To unsubscribe,

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95102/ Test PASSed. ---

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17400 **[Test build #95102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95102/testReport)** for PR 17400 at commit

[GitHub] spark issue #22185: [SPARK-25127] DataSourceV2: Remove SupportsPushDownCatal...

2018-08-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22185 +1 when tests pass. Convenient that there weren't any internal implementations using this. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22185: [SPARK-25127] DataSourceV2: Remove SupportsPushDownCatal...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22185 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22185: [SPARK-25127] DataSourceV2: Remove SupportsPushDownCatal...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22185 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22185: [SPARK-25127] DataSourceV2: Remove SupportsPushDownCatal...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22185 **[Test build #95110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95110/testReport)** for PR 22185 at commit

[GitHub] spark issue #22185: [SPARK-25127] DataSourceV2: Remove SupportsPushDownCatal...

2018-08-22 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/22185 cc @rdblue @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22185: [SPARK-25127] DataSourceV2: Remove SupportsPushDo...

2018-08-22 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/22185 [SPARK-25127] DataSourceV2: Remove SupportsPushDownCatalystFilters ## What changes were proposed in this pull request? They depend on internal Expression APIs. Let's see how far we can get

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread yifeih
Github user yifeih commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212017571 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -51,7 +57,13 @@

[GitHub] spark issue #22181: [SPARK-25163][SQL] Fix flaky test: o.a.s.util.collection...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22181: [SPARK-25163][SQL] Fix flaky test: o.a.s.util.collection...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95101/ Test PASSed. ---

[GitHub] spark issue #22181: [SPARK-25163][SQL] Fix flaky test: o.a.s.util.collection...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22181 **[Test build #95101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95101/testReport)** for PR 22181 at commit

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22009 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-08-22 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/22165 One general idea is that we don't need to rely on the RPC framework to test `ContextBarrierState`, just mock `RpcCallContext`s should be enough (haven't go into detail so correct me if I'm

[GitHub] spark issue #22183: [SPARK-25132][SQL][BACKPORT-2.3] Case-insensitive field ...

2018-08-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22183 See my comment: https://github.com/apache/spark/pull/22184/files#r212006137 --- - To unsubscribe, e-mail:

[GitHub] spark issue #22177: stages in wrong order within job page DAG chart

2018-08-22 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22177 Please change title to "[SPARK-25199][Web UI] XXX " as we described in http://spark.apache.org/contributing.html. ``` check the DAG chart in job page. ``` Could you also put

[GitHub] spark pull request #22184: [SPARK-25132][SQL][DOC] Add migration doc for cas...

2018-08-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22184#discussion_r212006137 --- Diff: docs/sql-programming-guide.md --- @@ -1895,6 +1895,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #95109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95109/testReport)** for PR 20637 at commit

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-22 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/22112 I'm not a fan of the IDEMPOTENT, RANDOM_ORDER, COMPLETE_RANDOM naming. IDEMPOTENT is fine, but I'd prefer UNORDERED and INDETERMINATE to cover the cases of "same values in potentially a

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22009 It sounds like the sync between apache and github is down. Although it has been merged, the PR has not been closed. --- -

[GitHub] spark pull request #22177: stages in wrong order within job page DAG chart

2018-08-22 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22177#discussion_r212003441 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala --- @@ -337,7 +337,9 @@ private[ui] class JobPage(parent: JobsTab, store:

[GitHub] spark pull request #22177: stages in wrong order within job page DAG chart

2018-08-22 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22177#discussion_r212002571 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala --- @@ -18,18 +18,18 @@ package org.apache.spark.ui.jobs import

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #95108 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95108/testReport)** for PR 21977 at commit

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread yifeih
Github user yifeih commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r211999160 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -51,7 +57,13 @@

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-08-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21912 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22179 Yes, I think the question is whether anything breaks as well. Kryo 4.x is a breaking change IIRC. Is there a 3.x version that has these fixes? ---

[GitHub] spark pull request #22180: [SPARK-25174][YARN]Limit the size of diagnostic m...

2018-08-22 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22180#discussion_r211996461 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -368,7 +369,11 @@ private[spark] class

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-22 Thread yifeih
Github user yifeih commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r211998423 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -96,3 +112,25 @@

[GitHub] spark pull request #22180: [SPARK-25174][YARN]Limit the size of diagnostic m...

2018-08-22 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22180#discussion_r211996874 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -143,6 +143,7 @@ private[spark] class

[GitHub] spark issue #22153: [SPARK-23034][SQL] Show RDD/relation names in RDD/In-Mem...

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22153 In the screenshot, do you know where we generate "in-memory table t [cached]"? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r211994713 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -853,33 +853,43 @@ case class

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r211994204 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -853,33 +853,43 @@ case class

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-08-22 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 Thanks @srowen [SPARK-25176](https://issues.apache.org/jira/browse/SPARK-25176) has a detail description: > I'm using the latest spark version spark-core_2.11:2.3.1 which transitively

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211992125 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1444,6 +1444,31 @@ object SQLConf { .intConf

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211991751 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -366,6 +366,43 @@ class AggregateBenchmark

[GitHub] spark issue #21749: [SPARK-24785] [SHELL] Making sure REPL prints Spark UI i...

2018-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21749 As it's more a cosmetic issue than anything, and reverting the Scala 2.12 change at least I think will be a problem, I'd suggest not reverting the scala version, but instead fixing forward. (Or at

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21931 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95107/ Test FAILed. ---

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21931 **[Test build #95107 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95107/testReport)** for PR 21931 at commit

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211990712 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1444,6 +1444,31 @@ object SQLConf { .intConf

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21931 **[Test build #95107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95107/testReport)** for PR 21931 at commit

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211989166 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- + +*

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211988217 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- + +*

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211987834 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- + +*

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21931 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211988526 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211987726 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-08-22 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r211986809 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -137,13 +137,21 @@ object RewritePredicateSubquery

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211986779 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211986370 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,377 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

<    1   2   3   4   5   6   7   >