[GitHub] spark issue #22343: [SPARK-25391][SQL] Make behaviors consistent when conver...

2018-09-11 Thread seancxmao
Github user seancxmao commented on the issue: https://github.com/apache/spark/pull/22343 Could we see this as a behavior change? We can add a legacy conf (e.g. `spark.sql.hive.legacy.convertMetastoreParquet`, may be defined in HiveUtils) to enable users to revert back to the previous

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r216600156 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22358 **[Test build #95930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95930/testReport)** for PR 22358 at commit [`dd86d3f`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22358 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22358 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95930/ Test PASSed. ---

[GitHub] spark pull request #22357: [SPARK-25363][SQL] Fix schema pruning in where cl...

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22357#discussion_r216601125 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -110,7 +110,17 @@ private[sql] object

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3008/

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22357 **[Test build #95931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95931/testReport)** for PR 22357 at commit [`d684a0f`](https://github.com/apache/spark/commit/d6

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attri...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216606555 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1153,12 +1154,35 @@ class FilterPushdo

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread Fokko
Github user Fokko commented on the issue: https://github.com/apache/spark/pull/21596 Some of the JSON endpoints are invalid, for example: ``` { "info": { "id": "driver", "hostPort": "localhost:61103", "isActive": true,

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread Fokko
Github user Fokko commented on the issue: https://github.com/apache/spark/pull/21596 Some of the code has been introduced here: https://github.com/apache/spark/commit/9241e1e7e66574cfafa68791771959dfc39c9684 --- - T

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 > Although event log is in JSON format, it's mostly for internal usage, to be load by history server and used to build the Spark UI. AFAIK, there are more and more projects replay event log to

[GitHub] spark issue #21791: [SPARK-24925][SQL] input bytesRead metrics fluctuate fro...

2018-09-11 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/21791 @kiszk I feel it is hard to add UT, do you have any idea? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22387 **[Test build #95925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95925/testReport)** for PR 22387 at commit [`a7b857c`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95932/testReport)** for PR 21596 at commit [`627f611`](https://github.com/apache/spark/commit/62

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95925/ Test FAILed. ---

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95932/testReport)** for PR 21596 at commit [`627f611`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95932/ Test FAILed. ---

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22387 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 Spark driver log is always distributed on various client nodes and depends on the log4j configs. In a big company, it's hard to collect them all and I think it's better to used for debug not analy

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22213 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3009/

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22387 **[Test build #95933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95933/testReport)** for PR 22387 at commit [`a7b857c`](https://github.com/apache/spark/commit/a7

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22213 **[Test build #95934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95934/testReport)** for PR 22213 at commit [`76635ce`](https://github.com/apache/spark/commit/76

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95935/testReport)** for PR 21596 at commit [`8c1da00`](https://github.com/apache/spark/commit/8c

[GitHub] spark pull request #22370: don't link to deprecated function

2018-09-11 Thread MichaelChirico
Github user MichaelChirico closed the pull request at: https://github.com/apache/spark/pull/22370 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95935/testReport)** for PR 21596 at commit [`8c1da00`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95935/ Test FAILed. ---

[GitHub] spark pull request #22393: Axe deprecated doc refs

2018-09-11 Thread MichaelChirico
GitHub user MichaelChirico opened a pull request: https://github.com/apache/spark/pull/22393 Axe deprecated doc refs Continuation of #22370. Summary of discussion there: There is some inconsistency in the R manual w.r.t. supercedent functions linking back to deprecated func

[GitHub] spark issue #22393: Axe deprecated doc refs

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22393 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22370: don't link to deprecated function

2018-09-11 Thread MichaelChirico
Github user MichaelChirico commented on the issue: https://github.com/apache/spark/pull/22370 @felixcheung thanks for the feedback, yes, I axed `registerTempTable` from the `SparkDataFrame `@family`. Sorry I botched an attempt to try and rename the branch of this PR (apparen

[GitHub] spark issue #21080: [SPARK-23986][SQL] freshName can generate non-unique nam...

2018-09-11 Thread dzanozin
Github user dzanozin commented on the issue: https://github.com/apache/spark/pull/21080 Please ignore the error report, it was a mess in the machine configuration with some v2.3.0 leftovers. Works as expected after cleanup. Sorry! ---

[GitHub] spark issue #22393: Axe deprecated doc refs

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22393 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22393: Axe deprecated doc refs

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22393 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22391 **[Test build #95927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95927/testReport)** for PR 22391 at commit [`66b6bd5`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95927/ Test FAILed. ---

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22391 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3010/

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22391 **[Test build #95936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95936/testReport)** for PR 22391 at commit [`66b6bd5`](https://github.com/apache/spark/commit/66

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22389 **[Test build #95924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95924/testReport)** for PR 22389 at commit [`9a04d87`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22389 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95924/ Test PASSed. ---

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22353 So you need a way to reliably report some extra information like file path in the event logs, but don't want to show it in the UI as it maybe too long. Basically we shouldn't put such info

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22341 **[Test build #95926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95926/testReport)** for PR 22341 at commit [`7c76790`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95926/ Test FAILed. ---

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22341 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3011/

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22341 **[Test build #95937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95937/testReport)** for PR 22341 at commit [`7c76790`](https://github.com/apache/spark/commit/7c

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attributes f...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22326 IIUC, you are pulling out the join condition with python UDF and create a filter above join. Then the join become a cross join, which usually runs very slowly. I think we should keep the cross joi

[GitHub] spark issue #22380: [SPARK-25278][SQL][followup] remove the hack in Progress...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22380 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22380: [SPARK-25278][SQL][followup] remove the hack in P...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22380 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216638725 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216638992 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 Thank you @cloud-fan for your reminding. We’ve handled the drop message case. Agree, I will update a commit tomorrow. --- -

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95938/testReport)** for PR 22337 at commit [`2d9e34a`](https://github.com/apache/spark/commit/2d

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3012/

[GitHub] spark issue #22374: [SPARK-25387][SQL] Fix for NPE caused by bad CSV input

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22374 **[Test build #95939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95939/testReport)** for PR 22374 at commit [`2a0dac4`](https://github.com/apache/spark/commit/2a

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95929/testReport)** for PR 21596 at commit [`98d459f`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95929/ Test FAILed. ---

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22389 Merged to master and branch-2.4. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95940/testReport)** for PR 21596 at commit [`f444a32`](https://github.com/apache/spark/commit/f4

[GitHub] spark pull request #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix em...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22389 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22234: [SPARK-25241][SQL] Configurable empty values when...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22234 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22367: [SPARK-17916][SPARK-25241][SQL][FOLLOWUP] Fix emp...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22367 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCode...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22358#discussion_r216657064 --- Diff: docs/sql-programming-guide.md --- @@ -965,6 +965,7 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22358 I'm okay but I would close this if no committer agree with (approves) this for some long time. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22357 **[Test build #95931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95931/testReport)** for PR 22357 at commit [`d684a0f`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22391 **[Test build #95936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95936/testReport)** for PR 22391 at commit [`66b6bd5`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22385 **[Test build #95928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95928/testReport)** for PR 22385 at commit [`daf76ed`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95931/ Test PASSed. ---

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95936/ Test FAILed. ---

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22385 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95928/ Test PASSed. ---

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22387 **[Test build #95933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95933/testReport)** for PR 22387 at commit [`a7b857c`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95933/ Test PASSed. ---

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22391 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3013/

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22391 **[Test build #95941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95941/testReport)** for PR 22391 at commit [`66b6bd5`](https://github.com/apache/spark/commit/66

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95942/testReport)** for PR 21596 at commit [`322ef7f`](https://github.com/apache/spark/commit/32

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95943/testReport)** for PR 21596 at commit [`278ff11`](https://github.com/apache/spark/commit/27

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216676386 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait ExpressionEvalHelpe

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216679499 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait ExpressionEvalHelpe

[GitHub] spark issue #22348: [SPARK-25354][SQL] Reduce unneeded operation in nextKeyV...

2018-09-11 Thread SongYadong
Github user SongYadong commented on the issue: https://github.com/apache/spark/pull/22348 @dongjoon-hyun . You are right, DataSourceReadBenchmark result show the benefit is too small even in some cases is covered up by fluctuation. Java HotSpot(TM) 64-Bit Server VM 1.8.0_141

[GitHub] spark pull request #22357: [SPARK-25363][SQL] Fix schema pruning in where cl...

2018-09-11 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/22357#discussion_r216683076 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -110,7 +110,17 @@ private[sql] objec

<    1   2   3   4   5   6   >