[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/22121 [SPARK-25133][SQL][Doc]AVRO data source guide ## What changes were proposed in this pull request? Create documentation for AVRO data source. The new page will be linked in https:

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2246/

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #94850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94850/testReport)** for PR 22121 at commit [`3d8220f`](https://github.com/apache/spark/commit/3d

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22117 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22117 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2247/

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22117 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22117 **[Test build #94851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94851/testReport)** for PR 22117 at commit [`3cad78f`](https://github.com/apache/spark/commit/3c

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #94850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94850/testReport)** for PR 22121 at commit [`3d8220f`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94850/ Test PASSed. ---

[GitHub] spark issue #22109: [SPARK-25120][CORE][HistoryServer]Fix the problem of Eve...

2018-08-16 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22109 can you please close this pr @deshanxiao ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22109: [SPARK-25120][CORE][HistoryServer]Fix the problem...

2018-08-16 Thread deshanxiao
Github user deshanxiao closed the pull request at: https://github.com/apache/spark/pull/22109 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-08-16 Thread redsanket
Github user redsanket commented on the issue: https://github.com/apache/spark/pull/21402 @Victsm @vanzin i want to get this going, is it better if I have a PR up with the requested changes and concerns? --- - To uns

[GitHub] spark pull request #21899: [SPARK-24912][SQL] Don't obscure source of OOM du...

2018-08-16 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21899#discussion_r210645167 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala --- @@ -118,12 +119,19 @@ case class BroadcastExchange

[GitHub] spark pull request #21899: [SPARK-24912][SQL] Don't obscure source of OOM du...

2018-08-16 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21899#discussion_r210647464 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala --- @@ -118,12 +119,19 @@ case class BroadcastExchange

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-08-16 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21899 lgtm, just a small reword in one of the msgs suggested by maxgekk. @hvanhovell @gatorsmile would you like to review this as well? --- ---

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-08-16 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21950 restest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in P...

2018-08-16 Thread xuanyuanking
GitHub user xuanyuanking opened a pull request: https://github.com/apache/spark/pull/22122 [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark to manage all sql configs ## What changes were proposed in this pull request? Follow up for SPARK-24665, find some others hard

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22122 **[Test build #94852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94852/testReport)** for PR 22122 at commit [`8a32e60`](https://github.com/apache/spark/commit/8a

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21950 **[Test build #94853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94853/testReport)** for PR 21950 at commit [`aa2a957`](https://github.com/apache/spark/commit/aa

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22122 Are they all instances to fix? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2248/

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22122 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22123: [SPARK-25134][SQL] Csv column pruning with checki...

2018-08-16 Thread koertkuipers
GitHub user koertkuipers opened a pull request: https://github.com/apache/spark/pull/22123 [SPARK-25134][SQL] Csv column pruning with checking of headers throws incorrect error ## What changes were proposed in this pull request? When column pruning is turned on the checking

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2249/

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22123 **[Test build #94854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94854/testReport)** for PR 22123 at commit [`c4179a9`](https://github.com/apache/spark/commit/c4

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-08-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21537 @HyukjinKwon I am worrying about the design of a mixture of representation s"" and code""? When the design is not good, it is hard to maintain it and add new code based on this. Let @cloud-fan to

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2250/

[GitHub] spark issue #22112: [WIP][SPARK-23243][Core] Fix RDD.repartition() data corr...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94855/testReport)** for PR 22112 at commit [`d187de8`](https://github.com/apache/spark/commit/d1

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22098 oh wow, nice find @HyukjinKwon! code archaeology ftw! :) anyways: that PR went through about a week after i started here @ the amplab and i don't think i even had access to the build s

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22112 > I would propose we fix the things that are using the round robin type partitioning (repartition) but then unordered things like zip/MapPartitions (via user code) we document or perhaps give the

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22122 **[Test build #94852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94852/testReport)** for PR 22122 at commit [`8a32e60`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22122 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22122: [SPARK-24665][PySpark][FollowUp] Use SQLConf in PySpark ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94852/ Test PASSed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2251/

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94856/testReport)** for PR 22112 at commit [`6f5d5e9`](https://github.com/apache/spark/commit/6f

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94855/testReport)** for PR 22112 at commit [`d187de8`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22101: [SPARK-25114][Core] Fix RecordBinaryComparator when subt...

2018-08-16 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22101 the added tests are good. This is pretty nit-picky, but looking at the whole test suite, are there any tests that check for anything other than the first byte (or array length)? Seems the longer ca

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94855/ Test FAILed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21899: [SPARK-24912][SQL] Don't obscure source of OOM du...

2018-08-16 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21899#discussion_r210664024 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala --- @@ -118,12 +119,19 @@ case class BroadcastExchang

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2252/

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94857/testReport)** for PR 22112 at commit [`2407328`](https://github.com/apache/spark/commit/24

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r210666117 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1492,6 +1492,15 @@ object SQLConf { "This usually

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94856/testReport)** for PR 22112 at commit [`6f5d5e9`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94856/ Test FAILed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94857/ Test FAILed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94857/testReport)** for PR 22112 at commit [`2407328`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22119: [WIP][SPARK-25129][SQL] Revert mapping com.databricks.sp...

2018-08-16 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22119 @tgravescs I saw your comments. Just feel that we can make it simpler by reverting it. For hive tables that used Databricks spark-avro, the tables can still using the Databricks repo(sinc

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2253/

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94858/testReport)** for PR 22112 at commit [`a43acdc`](https://github.com/apache/spark/commit/a4

[GitHub] spark issue #22119: [WIP][SPARK-25129][SQL] Revert mapping com.databricks.sp...

2018-08-16 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22119 But it seems that creating a configuration makes everyone happy... I will wait for another day to get more thoughts. --- --

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-08-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21537 @gatorsmile Thank you for your reply. Could you elaborate on your suggestion? >A general suggestion. To avoid introducing the regressions, how about implementing a new one without changing the exi

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94858/testReport)** for PR 22112 at commit [`a43acdc`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94858/ Test FAILed. ---

[GitHub] spark issue #22119: [WIP][SPARK-25129][SQL] Revert mapping com.databricks.sp...

2018-08-16 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22119 How do users manually migrate and keep compatibility?That is the problem I have, I am all for reverting, if we have an easy way for users to migrate to the internal one. Note that

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #94848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94848/testReport)** for PR 22104 at commit [`dcf07fb`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #20725: [SPARK-23555][PYTHON] Add BinaryType support for ...

2018-08-16 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20725#discussion_r210679722 --- Diff: python/pyspark/sql/tests.py --- @@ -4331,13 +4354,22 @@ def test_createDataFrame_fallback_enabled(self): self.assertEq

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94848/ Test PASSed. ---

[GitHub] spark issue #20725: [SPARK-23555][PYTHON] Add BinaryType support for Arrow i...

2018-08-16 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20725 @shaneknapp would you mind kicking off a test on one of the workers with python 3.5 and pyarrow 0.10.0 to verify this PR is passing under that env? Sorry for the special request, this should be

[GitHub] spark pull request #21858: [SPARK-24899][SQL][DOC] Add example of monotonica...

2018-08-16 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/21858#discussion_r210681673 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/MonotonicallyIncreasingID.scala --- @@ -80,7 +80,5 @@ case class Mon

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-16 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/22104 Tests pass now. This comment https://github.com/apache/spark/pull/22104/files#r210414941 requires some attention. @cloud-fan Do you think this is the right way to handle GenericInternalRow input

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-08-16 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r210690505 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -216,8 +217,7 @@ private[spark] class Executor( def stop(): Unit

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #94859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94859/testReport)** for PR 22121 at commit [`030ca0f`](https://github.com/apache/spark/commit/03

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21909 **[Test build #94860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94860/testReport)** for PR 21909 at commit [`6b34018`](https://github.com/apache/spark/commit/6b

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-08-16 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r210691276 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -69,6 +69,11 @@ package object config { .bytesConf(ByteUnit.Ki

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2254/

[GitHub] spark issue #22120: [SPARK-25131]Event logs missing applicationAttemptId for...

2018-08-16 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22120 Is this really necessary? It will always be "1", since client-mode apps are not re-tried (the YARN AM might be, but the driver is not). That makes it not really useful. --- ---

[GitHub] spark issue #20725: [SPARK-23555][PYTHON] Add BinaryType support for Arrow i...

2018-08-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/20725 https://rise.cs.berkeley.edu/jenkins/job/ubuntuSparkPRB/79/ --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r210693829 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2223,21 +2223,31 @@ class JsonSuite extends Qu

[GitHub] spark issue #20725: [SPARK-23555][PYTHON] Add BinaryType support for Arrow i...

2018-08-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/20725 ugh hang on. lemme fix that crap. (this is on a testing branch in my personal spark form) --- - To unsubscribe, e-mail: rev

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #94859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94859/testReport)** for PR 22121 at commit [`030ca0f`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94859/ Test PASSed. ---

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]AVRO data source guide

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20725: [SPARK-23555][PYTHON] Add BinaryType support for Arrow i...

2018-08-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/20725 https://rise.cs.berkeley.edu/jenkins/job/ubuntuSparkPRB/80/ --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20725: [SPARK-23555][PYTHON] Add BinaryType support for Arrow i...

2018-08-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/20725 i've been having trouble getting this build to pass, and i'm testing a few other modules on top of yours. if i have problems, i'll revert the extraneous changes and build yours. --- -

[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22081 **[Test build #94849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94849/testReport)** for PR 22081 at commit [`d0334c1`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94849/ Test PASSed. ---

[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22081 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22124: [SPARK-25135][SQL] Insert datasource table may al...

2018-08-16 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22124 [SPARK-25135][SQL] Insert datasource table may all null when select from view ## What changes were proposed in this pull request? How to reproduce: ```scala val path = "/tmp/spark/

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2255/

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22124 **[Test build #94861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94861/testReport)** for PR 22124 at commit [`276879c`](https://github.com/apache/spark/commit/27

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22117 **[Test build #94851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94851/testReport)** for PR 22117 at commit [`3cad78f`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22117 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22117 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94851/ Test FAILed. ---

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #21819: [SPARK-24863][SS] Report Kafka offset lag as a custom me...

2018-08-16 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21819 @HyukjinKwon , can you take it forward? Appreciate your effort and thanks in advance. --- - To unsubscribe, e-mail: revie

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-16 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r210704902 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2223,21 +2223,31 @@ class JsonSuite extends Query

<    1   2   3   4   >