[GitHub] spark issue #22382: [SPARK-23243] [SPARK-20715][CORE][2.2] Fix RDD.repartiti...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22382 Since it's merged, could you close this PR, @bersprockets ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.

[GitHub] spark issue #13893: [SPARK-14172][SQL] Hive table partition predicate not pa...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13893 Ping, @jiangxb1987 . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22211 Hi, @henryr . Since this is merged, could you close this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95917/testReport)** for PR 22337 at commit [`2d9e34a`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22389 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95915/testReport)** for PR 22192 at commit [`447c5e5`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22389 **[Test build #95922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95922/testReport)** for PR 22389 at commit [`9a04d87`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 **[Test build #95913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95913/testReport)** for PR 22376 at commit [`4a0cffb`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95919/testReport)** for PR 21596 at commit [`bb8b661`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22389 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95922/ Test FAILed. ---

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95918/testReport)** for PR 7 at commit [`b12ee88`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22385 **[Test build #95914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95914/testReport)** for PR 22385 at commit [`daf76ed`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95919/ Test FAILed. ---

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22387 **[Test build #95921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95921/testReport)** for PR 22387 at commit [`a7b857c`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22213 **[Test build #95920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95920/testReport)** for PR 22213 at commit [`76635ce`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22213 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95920/ Test FAILed. ---

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22192 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22385 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22213 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22192 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95915/ Test FAILed. ---

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/7 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95918/ Test FAILed. ---

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95921/ Test FAILed. ---

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22385 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95914/ Test FAILed. ---

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95917/ Test FAILed. ---

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/7 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95913/ Test FAILed. ---

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22390 [SPARK-25402][SQL] Null handling in BooleanSimplification ## What changes were proposed in this pull request? This PR is to fix the null handling in BooleanSimplification. In the rule Boolea

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22390 cc @cloud-fan @adrian-ionescu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3002/

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22389 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22390 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22390 **[Test build #95923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95923/testReport)** for PR 22390 at commit [`cf863bb`](https://github.com/apache/spark/commit/cf

[GitHub] spark issue #22389: [SPARK-17916][SPARK-25241][SQL][FOLLOW-UP] Fix empty str...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22389 **[Test build #95924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95924/testReport)** for PR 22389 at commit [`9a04d87`](https://github.com/apache/spark/commit/9a

[GitHub] spark pull request #22357: [SPARK-25363][SQL] Fix schema pruning in where cl...

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22357#discussion_r216565635 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -110,7 +110,17 @@ private[sql] object

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22387 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-09-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 I will close it now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3003/

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22388 yes --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apac

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22387 **[Test build #95925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95925/testReport)** for PR 22387 at commit [`a7b857c`](https://github.com/apache/spark/commit/a7

[GitHub] spark pull request #22374: [SPARK-25387][SQL] Fix for NPE caused by bad CSV ...

2018-09-11 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22374#discussion_r216569630 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1700,4 +1700,13 @@ class CSVSuite extends QueryTest

[GitHub] spark pull request #22364: [SPARK-25379][SQL] Improve AttributeSet and Colum...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22364#discussion_r216572034 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala --- @@ -39,10 +41,15 @@ object AttributeSet {

[GitHub] spark pull request #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to su...

2018-09-11 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20504 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22341: [SPARK-24889][Core] Update block info when unpers...

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22341#discussion_r216574255 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -646,7 +647,47 @@ private[spark] class AppStatusListener( }

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3004/

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22341 @vanzin Thanks for the good review! I've updated this to address them all. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22341 **[Test build #95926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95926/testReport)** for PR 22341 at commit [`7c76790`](https://github.com/apache/spark/commit/7c

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216575397 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -263,10 +263,12 @@ object BooleanSimplification ex

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216575336 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala --- @@ -48,6 +48,14 @@ class BooleanSi

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216574955 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala --- @@ -48,6 +48,14 @@ class BooleanSi

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216576135 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -263,10 +263,12 @@ object BooleanSimplificatio

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95923/ Test FAILed. ---

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22390 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22390 **[Test build #95923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95923/testReport)** for PR 22390 at commit [`cf863bb`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22343: [SPARK-25391][SQL] Make behaviors consistent when conver...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22343 To clarify: this is just a workaround when we hit a problematic(having case-insensitive duplicated filed names in the parquet file) hive parquet tables and we want to read it with the native parqu

[GitHub] spark issue #22371: [SPARK-25386][CORE] Don't need to synchronize the IndexS...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22371 @ConeyLiu we may have an executor lost and then come back, and may have 2 same tasks running on the same executor. --- - To un

[GitHub] spark issue #22343: [SPARK-25391][SQL] Make behaviors consistent when conver...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22343 Thank you for the pointer, @seancxmao . And thank you for clarification, @cloud-fan . It looks like we are re-creating correctness issue somewhat in this PR when `caseSensitive=true`.

[GitHub] spark issue #22371: [SPARK-25386][CORE] Don't need to synchronize the IndexS...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22371 My opinion is, it's not worth to spend time on it. The lock is not likely to be a bottleneck and it's better to keep it simple even it's sub-optimal. ---

[GitHub] spark pull request #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should ...

2018-09-11 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/22391 [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow being called with 0 args ## What changes were proposed in this pull request? SPARK-21281 introduced a check for the inputs of `Creat

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22391 cc @cloud-fan @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22378: [SPARK-25389][SQL] INSERT OVERWRITE DIRECTORY STO...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22378#discussion_r216580577 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -750,4 +751,27 @@ class InsertSuite extends QueryTest with TestHiveS

[GitHub] spark issue #22378: [SPARK-25389][SQL] INSERT OVERWRITE DIRECTORY STORED AS ...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22378 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3005/

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22391 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22369: [SPARK-25072][DOC] Update migration guide for behavior c...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22369 As the comment in https://github.com/apache/spark/pull/22140#issuecomment-419997180, I think this doc change is no more needed, I just close this, thanks @BryanCutler and @HyukjinKwon ! ---

[GitHub] spark pull request #22369: [SPARK-25072][DOC] Update migration guide for beh...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking closed the pull request at: https://github.com/apache/spark/pull/22369 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22391 **[Test build #95927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95927/testReport)** for PR 22391 at commit [`66b6bd5`](https://github.com/apache/spark/commit/66

[GitHub] spark issue #22367: [SPARK-17916][SPARK-25241][SQL][FOLLOWUP] Fix empty stri...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22367 Usually we merge into master and backport to other branches when it's needed. https://spark.apache.org/contributing.html > 5. Open a pull request against the master branch of ap

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attri...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216583509 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1153,12 +1154,35 @@ class FilterPushdo

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attri...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216585000 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1153,12 +1154,35 @@ class FilterPushdo

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22357 Can anyone point me out if there are non addressed comments or problems here? Looks pretty good to me. I think this is rather a bandaid, small and safe fix to get into branch-2.4. --- ---

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-11 Thread ssaavedra
GitHub user ssaavedra opened a pull request: https://github.com/apache/spark/pull/22392 [SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore Several configuration parameters related to Kubernetes need to be reset, as they are changed with each invokation of spark-

[GitHub] spark issue #22392: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22392 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22371: [SPARK-25386][CORE] Don't need to synchronize the...

2018-09-11 Thread ConeyLiu
Github user ConeyLiu closed the pull request at: https://github.com/apache/spark/pull/22371 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22385 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22371: [SPARK-25386][CORE] Don't need to synchronize the IndexS...

2018-09-11 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/22371 OK, thanks everyone for the help. Close it --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r216587584 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkHadoopUtilSuite.scala --- @@ -77,6 +80,51 @@ class SparkHadoopUtilSuite extends SparkFunSuit

[GitHub] spark issue #22392: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22392 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22392: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22392 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3006/

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22385 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22385: [SPARK-25400][CORE] Increase test timeouts

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22385 **[Test build #95928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95928/testReport)** for PR 22385 at commit [`daf76ed`](https://github.com/apache/spark/commit/da

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r216588353 --- Diff: core/src/main/java/org/apache/hadoop/fs/SparkGlobber.java --- @@ -0,0 +1,293 @@ +/** + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread adrian-ionescu
Github user adrian-ionescu commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216587101 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -263,10 +263,12 @@ object BooleanSimplificati

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread adrian-ionescu
Github user adrian-ionescu commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216587787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -263,10 +263,12 @@ object BooleanSimplificati

[GitHub] spark issue #21080: [SPARK-23986][SQL] freshName can generate non-unique nam...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21080 @dzanozin the code you reported clearly misses this patch. Since I do see the commit on v2.3.1, I am a bit puzzled about the issue you reported. Please provide a reproducer so we can check whether

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95929/testReport)** for PR 21596 at commit [`98d459f`](https://github.com/apache/spark/commit/98

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attri...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216592676 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1153,12 +1154,35 @@ class FilterPushdownSu

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216593044 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) exten

[GitHub] spark issue #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Support par...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20999 cc @cloud-fan @dongjoon-hyun too, in case have time to take a look and review this, thanks. --- - To unsubscribe, e-mail: revie

[GitHub] spark pull request #22374: [SPARK-25387][SQL] Fix for NPE caused by bad CSV ...

2018-09-11 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22374#discussion_r216593835 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -243,17 +243,21 @@ object TextInputCSVDat

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22358 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3007/

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22358 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22358 **[Test build #95930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95930/testReport)** for PR 22358 at commit [`dd86d3f`](https://github.com/apache/spark/commit/dd

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2018-09-11 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r216597619 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -373,8 +373,14 @@ private[spark] class ExecutorAllocationManager(

  1   2   3   4   5   6   >