[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95258/testReport)** for PR 22197 at commit

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95257/testReport)** for PR 22197 at commit

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 @gatorsmile I can help check `spark.sql.caseSensitive` for all the built-in data sources. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212814524 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -350,25 +356,38 @@ private[parquet] class

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 Thanks, @dongjoon-hyun! I'll check and merge that. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212814249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -44,7 +45,12 @@ private[parquet] class

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22197 This PR is basically trying to resolve case sensitivity when the logical schema and physical schema do not match. This sounds like a general issue in all the data sources. Could any of you do us

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212814157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -350,25 +356,38 @@ private[parquet]

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22197 @dongjoon-hyun Do you think we face the same issue in ORC? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r212813730 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95256/testReport)** for PR 22197 at commit

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212813302 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1022,116 @@ class

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212813240 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1022,116 @@ class

[GitHub] spark issue #20901: [SPARK-23792][DOCS] Documentation improvements for datet...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20901 **[Test build #4292 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4292/testReport)** for PR 20901 at commit

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212812744 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1022,116 @@ class

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212812771 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -44,7 +45,12 @@ private[parquet] class

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212812718 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1022,116 @@ class

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212812673 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -350,25 +356,38 @@ private[parquet] class

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212812755 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1022,116 @@ class

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-08-25 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r212812762 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile --- @@ -43,6 +43,10 @@ COPY sbin /opt/spark/sbin COPY

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21638 **[Test build #4290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4290/testReport)** for PR 21638 at commit

[GitHub] spark issue #22063: [WIP][SPARK-25044][SQL] Address translation of LMF closu...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22063 **[Test build #4289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4289/testReport)** for PR 22063 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22198 @maropu . I made a PR (https://github.com/maropu/spark/pull/2) for the new test cases. You can review and merge that if you want. `GlobalTempViewSuite.scala` will start to fail after merging.

[GitHub] spark pull request #16677: [SPARK-19355][SQL] Use map output statistics to i...

2018-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16677#discussion_r212811618 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -93,25 +96,93 @@ trait BaseLimitExec extends UnaryExecNode with

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212811422 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,39 @@ object ResolveHints {

[GitHub] spark issue #20901: [SPARK-23792][DOCS] Documentation improvements for datet...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20901 **[Test build #4292 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4292/testReport)** for PR 20901 at commit

[GitHub] spark issue #20901: [SPARK-23792][DOCS] Documentation improvements for datet...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20901 The failure isn't related: ``` [error] (kubernetes-integration-tests/*:checkstyle) java.io.FileNotFoundException: checkstyle-config.xml (No such file or directory) [error] Total

[GitHub] spark issue #20901: [SPARK-23792][DOCS] Documentation improvements for datet...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20901 **[Test build #4291 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4291/testReport)** for PR 20901 at commit

[GitHub] spark issue #20901: [SPARK-23792][DOCS] Documentation improvements for datet...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20901 **[Test build #4291 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4291/testReport)** for PR 20901 at commit

[GitHub] spark issue #21524: [SPARK-24212][ML][doc] Add the example and user guide fo...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21524 OK I guess this is just copied from the .mllib example and docs? I guess that's OK as this was done for other docs/examples. There's a Python API for this one too, it seems; how about a quick Python

[GitHub] spark issue #21664: [SPARK-24687][CORE] NoClassDefFoundError will not be cat...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21664 @caneGuy can you help me see why returning from this method aborts the job but throwing an error doesn't? If so, yeah that's not good. Why not just catch all exceptions instead of NonFatal only?

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21638 **[Test build #4290 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4290/testReport)** for PR 21638 at commit

[GitHub] spark issue #22063: [WIP][SPARK-25044][SQL] Address translation of LMF closu...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22063 **[Test build #4289 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4289/testReport)** for PR 22063 at commit

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212807892 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,39 @@ object ResolveHints {

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212807880 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -144,7 +144,7 @@ class Analyzer(

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212807800 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameJoinSuite.scala --- @@ -191,6 +195,39 @@ class DataFrameJoinSuite extends QueryTest

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22162 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22162 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95254/ Test PASSed. ---

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22162 **[Test build #95254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95254/testReport)** for PR 22162 at commit

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22197 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22197 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95253/ Test PASSed. ---

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95253/testReport)** for PR 22197 at commit

[GitHub] spark issue #22063: [WIP][SPARK-25044][SQL] Address translation of LMF closu...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22063 **[Test build #4288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4288/testReport)** for PR 22063 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95252/ Test PASSed. ---

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22198 **[Test build #95252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95252/testReport)** for PR 22198 at commit

[GitHub] spark pull request #16677: [SPARK-19355][SQL] Use map output statistics to i...

2018-08-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16677#discussion_r212805707 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -93,25 +96,93 @@ trait BaseLimitExec extends UnaryExecNode with

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-25 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212805339 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #16677: [SPARK-19355][SQL] Use map output statistics to i...

2018-08-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16677#discussion_r212805327 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -93,25 +96,93 @@ trait BaseLimitExec extends UnaryExecNode with

[GitHub] spark issue #22234: [SPARK-25241][SQL] Configurable empty values when readin...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22234 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22234: [SPARK-25241][SQL] Configurable empty values when readin...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22234 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22234: [SPARK-25241][SQL] Configurable empty values when readin...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22234 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22234: [SPARK-25241][SQL] Configurable empty values when...

2018-08-25 Thread mmolimar
GitHub user mmolimar opened a pull request: https://github.com/apache/spark/pull/22234 [SPARK-25241][SQL] Configurable empty values when reading/writing CSV files ## What changes were proposed in this pull request? There is an option in the CSV parser to set values when we have

[GitHub] spark issue #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITI...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22233 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITI...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22233 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95251/ Test PASSed. ---

[GitHub] spark issue #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITI...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22233 **[Test build #95251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95251/testReport)** for PR 22233 at commit

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-25 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r212804252 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1131,7 +1135,7 @@ abstract class DDLSuite extends

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread yifeih
Github user yifeih commented on the issue: https://github.com/apache/spark/pull/22146 Looking at this https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2256/console, I don't think my integration test ran... am I missing a change somewhere?

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2554/ ---

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95255/ Test PASSed. ---

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 **[Test build #95255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95255/testReport)** for PR 22146 at commit

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2554/ ---

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22146 **[Test build #95255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95255/testReport)** for PR 22146 at commit

[GitHub] spark pull request #22230: [SPARK-25214][SS][FOLLOWUP]Fix the issue that Kaf...

2018-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22230 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22042 > This patch fails Spark unit tests. This is the flaky test I fixed in #22230 retest this please --- - To

[GitHub] spark issue #22230: [SPARK-25214][SS][FOLLOWUP]Fix the issue that Kafka v2 s...

2018-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22230 Thanks! Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r212802305 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1131,7 +1135,7 @@ abstract class DDLSuite extends

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22162 **[Test build #95254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95254/testReport)** for PR 22162 at commit

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r212801959 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql](

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22162 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95253/testReport)** for PR 22197 at commit

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212801680 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1021,88 @@ class

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-25 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 @cloud-fan @HyukjinKwon Seem cannot simply add `originalName` into `ParquetSchemaType`. Because we need exact ParquetSchemaType info for type match, like: ``` private val

[GitHub] spark issue #22063: [WIP][SPARK-25044][SQL] Address translation of LMF closu...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22063 **[Test build #4288 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4288/testReport)** for PR 22063 at commit

[GitHub] spark pull request #22063: [WIP][SPARK-25044][SQL] Address translation of LM...

2018-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22063#discussion_r212800597 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3713,7 +3726,7 @@ object functions { | */ |def

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22193 @rdblue, is it a mistake to include the commits for SPARK-25188? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22198 **[Test build #95252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95252/testReport)** for PR 22198 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #21330: [SPARK-22234] Support distinct window functions

2018-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21330#discussion_r212800158 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1883,7 +1883,19 @@ class Analyzer( //

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-25 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212799852 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #21665: [SPARK-24688][examples]Modify the comments about ...

2018-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21665 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21665: [SPARK-24688][examples]Modify the comments about Labeled...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21665 Actually, the other cases aren't referring to a LabeledPoint, so I'll leave them. This is fine, and I'll merge it. --- - To

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212799711 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -44,6 +44,12 @@ class

[GitHub] spark pull request #22224: [SPARK-24090][K8S] Update running-on-kubernetes.m...

2018-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4#discussion_r212799656 --- Diff: docs/running-on-kubernetes.md --- @@ -185,6 +185,35 @@ To use a secret through an environment variable use the following options to the

[GitHub] spark pull request #22224: [SPARK-24090][K8S] Update running-on-kubernetes.m...

2018-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4#discussion_r212799672 --- Diff: docs/running-on-kubernetes.md --- @@ -299,21 +328,15 @@ RBAC authorization and how to configure Kubernetes service accounts for pods, pl

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r212799646 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -17,11 +17,13 @@ package org.apache.spark.executor

[GitHub] spark pull request #22222: [SPARK-25083][SQL] Remove the type erasure hack i...

2018-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2#discussion_r212799628 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -307,7 +308,7 @@ case class FileSourceScanExec(

[GitHub] spark issue #21665: [SPARK-24688][examples]Modify the comments about Labeled...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21665 **[Test build #4287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4287/testReport)** for PR 21665 at commit

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22231 Interesting, errors like: ``` ./dev/run-tests.py:172:7: W605 invalid escape sequence '\d' ``` on lines like: ``` match = re.search('(\d+)\.(\d+)\.(\d+)',

[GitHub] spark issue #21665: [SPARK-24688][examples]Modify the comments about Labeled...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21665 **[Test build #4287 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4287/testReport)** for PR 21665 at commit

[GitHub] spark issue #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITI...

2018-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22233 **[Test build #95251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95251/testReport)** for PR 22233 at commit

[GitHub] spark issue #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITI...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22233 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITI...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22233 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22232 It's OK to assume Hadoop 2.6+ only. In fact 2.6 is quite old anyway. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-25 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/22233 [SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITIONS ## What changes were proposed in this pull request? In the PR, I propose to not perform recursive parallel listening of files

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-25 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r212798970 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -350,25 +352,43 @@ private[parquet] class

  1   2   >