[GitHub] [spark] HyukjinKwon closed pull request #31361: [SPARK-34262][SQL] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
HyukjinKwon closed pull request #31361: URL: https://github.com/apache/spark/pull/31361 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] LuciferYang opened a new pull request #31376: [SPARK-34275][CORE][SQL][MLLIB][2.4] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang opened a new pull request #31376: URL: https://github.com/apache/spark/pull/31376 ### What changes were proposed in this pull request? Use `count` to simplify `find + size(or length)` operation, it's semantically consistent, but looks simpler. **Before** ```

[GitHub] [spark] sunchao commented on a change in pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
sunchao commented on a change in pull request #31355: URL: https://github.com/apache/spark/pull/31355#discussion_r565840043 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/distributions/OrderedDistribution.java ## @@ -32,4 +32,13 @@ * Returns

[GitHub] [spark] SparkQA commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768820326 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39171/

[GitHub] [spark] gengliangwang commented on a change in pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
gengliangwang commented on a change in pull request #31349: URL: https://github.com/apache/spark/pull/31349#discussion_r565838305 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala ## @@ -0,0 +1,305 @@ +/* + * Licensed to

[GitHub] [spark] LuciferYang opened a new pull request #31375: [SPARK-34275][CORE][SQL][MLLIB][3.0] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang opened a new pull request #31375: URL: https://github.com/apache/spark/pull/31375 ### What changes were proposed in this pull request? Use `count` to simplify `find + size(or length)` operation, it's semantically consistent, but looks simpler. **Before** ```

[GitHub] [spark] SparkQA commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
SparkQA commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768818290 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39173/

[GitHub] [spark] sunchao commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
sunchao commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768817191 @cloud-fan sure - I can reopen it later to include more test coverage for this. This is an automated message

[GitHub] [spark] SparkQA commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
SparkQA commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768816190 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39176/

[GitHub] [spark] dongjoon-hyun commented on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
dongjoon-hyun commented on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768816010 +1, late LGTM. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] gengliangwang commented on a change in pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
gengliangwang commented on a change in pull request #31349: URL: https://github.com/apache/spark/pull/31349#discussion_r565834360 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala ## @@ -0,0 +1,305 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768814873 @sunchao it's still valuable to keep your PR and add tests :) This is an automated message from the Apache

[GitHub] [spark] sunchao closed pull request #31182: [SPARK-34108][SQL] Caching with permanent view doesn't work in certain cases

2021-01-27 Thread GitBox
sunchao closed pull request #31182: URL: https://github.com/apache/spark/pull/31182 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] sunchao commented on pull request #31182: [SPARK-34108][SQL] Caching with permanent view doesn't work in certain cases

2021-01-27 Thread GitBox
sunchao commented on pull request #31182: URL: https://github.com/apache/spark/pull/31182#issuecomment-768813632 Closing this as it is duplicated by #31368 This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768796533 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39167/

[GitHub] [spark] sunchao commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
sunchao commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768813235 @cloud-fan it's working now - thanks! I'll close the JIRA as duplicate. This is an automated message from the

[GitHub] [spark] LuciferYang edited a comment on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang edited a comment on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768812793 > Yeah, let's do that. When we fix such kind of things, the conflicts are one of the usual problems pointed out by people. It should be better to port them back.

[GitHub] [spark] SparkQA commented on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
SparkQA commented on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768812938 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39174/

[GitHub] [spark] LuciferYang edited a comment on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang edited a comment on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768812492 > BTW, they are all instances, right? Yes This is an automated message from the Apache Git

[GitHub] [spark] LuciferYang commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768812793 > Yeah, let's do that. When we fix such kind of things, the conflicts are one of the usual problems pointed out by people. It should be better to port them back. OK

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768796536 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134587/

[GitHub] [spark] dongjoon-hyun closed pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
dongjoon-hyun closed pull request #31365: URL: https://github.com/apache/spark/pull/31365 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] LuciferYang commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768812492 > BTW, they are all instances, right? Yes This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768812256 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39175/

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768812243 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39175/

[GitHub] [spark] AmplabJenkins commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768812256 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39175/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768810770 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134575/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768810775 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39168/

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768810775 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39168/

[GitHub] [spark] AmplabJenkins commented on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768810770 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134575/

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768810389 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39175/

[GitHub] [spark] SparkQA commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
SparkQA commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768809480 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39173/

[GitHub] [spark] gengliangwang commented on a change in pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
gengliangwang commented on a change in pull request #31349: URL: https://github.com/apache/spark/pull/31349#discussion_r565828667 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala ## @@ -0,0 +1,305 @@ +/* + * Licensed to

[GitHub] [spark] gengliangwang commented on a change in pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
gengliangwang commented on a change in pull request #31349: URL: https://github.com/apache/spark/pull/31349#discussion_r565828667 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala ## @@ -0,0 +1,305 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768808211 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39171/

[GitHub] [spark] gengliangwang commented on a change in pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
gengliangwang commented on a change in pull request #31349: URL: https://github.com/apache/spark/pull/31349#discussion_r565827505 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala ## @@ -0,0 +1,305 @@ +/* + * Licensed to

[GitHub] [spark] beliefer commented on pull request #31312: [SPARK-33542][SQL][FOLLOWUP] Group exception messages in catalyst/catalog

2021-01-27 Thread GitBox
beliefer commented on pull request #31312: URL: https://github.com/apache/spark/pull/31312#issuecomment-768805556 @allisonwang-db @cloud-fan Thanks a lot! This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] cloud-fan commented on pull request #31312: [SPARK-33542][SQL][FOLLOWUP] Group exception messages in catalyst/catalog

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31312: URL: https://github.com/apache/spark/pull/31312#issuecomment-768805285 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #31312: [SPARK-33542][SQL][FOLLOWUP] Group exception messages in catalyst/catalog

2021-01-27 Thread GitBox
cloud-fan closed pull request #31312: URL: https://github.com/apache/spark/pull/31312 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] ulysses-you commented on a change in pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
ulysses-you commented on a change in pull request #31372: URL: https://github.com/apache/spark/pull/31372#discussion_r565825593 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala ## @@ -125,20 +125,34 @@ package object util extends

[GitHub] [spark] cloud-fan closed pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

2021-01-27 Thread GitBox
cloud-fan closed pull request #31341: URL: https://github.com/apache/spark/pull/31341 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768804331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565824188 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1763,6 +1763,21 @@ class Analyzer(override val

[GitHub] [spark] HyukjinKwon commented on pull request #31370: [SPARK-34268][SQL][DOCS] Correct the documentation of the concat_ws function

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31370: URL: https://github.com/apache/spark/pull/31370#issuecomment-768802709 Merged to master, branch-3.1, branch-3.0 and branch-2.4. This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon closed pull request #31370: [SPARK-34268][SQL][DOCS] Correct the documentation of the concat_ws function

2021-01-27 Thread GitBox
HyukjinKwon closed pull request #31370: URL: https://github.com/apache/spark/pull/31370 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HeartSaVioR commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
HeartSaVioR commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768802431 Yes thanks for confirming. I wanted to make sure my understanding about possible behavioral change is correct. I agree that is not related to this PR.

[GitHub] [spark] HyukjinKwon commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768802126 BTW, they are all instances, right? This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] cloud-fan commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768802108 It should keep the user intention (may need to fix some bugs). But this seems unrelated to this PR. This is

[GitHub] [spark] HyukjinKwon commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768801951 Yeah, let's do that. When we fix such kind of things, the conflicts are one of the usual problems pointed out by people. It should be better to port them back.

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
HeartSaVioR edited a comment on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768801016 > AQE won't kick in if users specify num partitions, e.g. df.repartition(5), I think the same applies here if the sink requires a certain num partitions. The

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
HyukjinKwon commented on a change in pull request #31372: URL: https://github.com/apache/spark/pull/31372#discussion_r565822721 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala ## @@ -125,20 +125,34 @@ package object util extends

[GitHub] [spark] cloud-fan commented on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768801329 @maropu the PR for master is merged: https://github.com/apache/spark/pull/31360 This is an automated message

[GitHub] [spark] HeartSaVioR commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
HeartSaVioR commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768801016 > AQE won't kick in if users specify num partitions, e.g. df.repartition(5), I think the same applies here if the sink requires a certain num partitions. The case I

[GitHub] [spark] HyukjinKwon closed pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
HyukjinKwon closed pull request #31371: URL: https://github.com/apache/spark/pull/31371 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768800479 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun closed pull request #31360: [SPARK-34260][SQL] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
dongjoon-hyun closed pull request #31360: URL: https://github.com/apache/spark/pull/31360 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768799766 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39168/

[GitHub] [spark] SparkQA removed a comment on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
SparkQA removed a comment on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768757650 **[Test build #134575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134575/testReport)** for PR 31371 at commit

[GitHub] [spark] SparkQA commented on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
SparkQA commented on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768799538 **[Test build #134575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134575/testReport)** for PR 31371 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565820592 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTablesSuite.scala ## @@ -34,7 +35,7 @@ trait ShowTablesSuiteBase

[GitHub] [spark] cloud-fan commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565819837 ## File path: docs/sql-migration-guide.md ## @@ -40,6 +40,10 @@ license: | - In Spark 3.2, script transform default FIELD DELIMIT is `\u0001` for no

[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
linhongliu-db commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565819800 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1763,6 +1763,21 @@ class Analyzer(override

[GitHub] [spark] cloud-fan commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565819393 ## File path: docs/sql-migration-guide.md ## @@ -40,6 +40,10 @@ license: | - In Spark 3.2, script transform default FIELD DELIMIT is `\u0001` for no

[GitHub] [spark] wangyum commented on pull request #31370: [SPARK-34268][SQL][DOCS] Correct the documentation of the concat_ws function

2021-01-27 Thread GitBox
wangyum commented on pull request #31370: URL: https://github.com/apache/spark/pull/31370#issuecomment-768797866 @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768797232 AQE won't kick in if users specify num partitions, e.g. `df.repartition(5)`, I think the same applies here if the sink requires a certain num partitions.

[GitHub] [spark] AmplabJenkins commented on pull request #31370: [SPARK-34268][SQL][DOCS] Correct the documentation of the concat_ws function

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31370: URL: https://github.com/apache/spark/pull/31370#issuecomment-768796531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134574/

[GitHub] [spark] AmplabJenkins commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768796536 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134587/

[GitHub] [spark] AmplabJenkins commented on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768796533 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39167/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768796287 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39169/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768796318 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39166/

[GitHub] [spark] AmplabJenkins commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768796318 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39166/

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

2021-01-27 Thread GitBox
SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768796275 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39169/

[GitHub] [spark] AmplabJenkins commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768796287 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39169/

[GitHub] [spark] SparkQA commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
SparkQA commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768796298 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39166/

[GitHub] [spark] SparkQA commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
SparkQA commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768795880 **[Test build #134585 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134585/testReport)** for PR 31372 at commit

[GitHub] [spark] cloud-fan commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768795364 @sunchao I fixed some places, can you try again? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
SparkQA commented on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768795142 **[Test build #134586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134586/testReport)** for PR 31356 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565816065 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala ## @@ -17,74 +17,21 @@ package

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768794925 **[Test build #134588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134588/testReport)** for PR 31368 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768794138 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39170/

[GitHub] [spark] AmplabJenkins commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768794138 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39170/

[GitHub] [spark] SparkQA commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768794126 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39170/

[GitHub] [spark] LuciferYang commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768793669 If need other PR for branch-3.0 and branch-2.4, please ping me This is an automated message from the Apache

[GitHub] [spark] wangyum commented on pull request #26804: [SPARK-26346][BUILD][SQL] Upgrade Parquet to 1.11.1

2021-01-27 Thread GitBox
wangyum commented on pull request #26804: URL: https://github.com/apache/spark/pull/26804#issuecomment-768793621 > Could you create a 3.2.0 blocker JIRA? OK, https://issues.apache.org/jira/browse/SPARK-34276. This is

[GitHub] [spark] LuciferYang opened a new pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang opened a new pull request #31374: URL: https://github.com/apache/spark/pull/31374 ### What changes were proposed in this pull request? Use `count` to simplify `find + size(or length)` operation, it's semantically consistent, but looks simpler. **Before** ```

[GitHub] [spark] SparkQA commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768792794 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39170/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768792501 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39162/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768792500 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39161/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768792499 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39165/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31354: [SPARK-34256][ML] VectorSlicer refine numFeatures checking and toString method

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31354: URL: https://github.com/apache/spark/pull/31354#issuecomment-768792498 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39163/

[GitHub] [spark] SparkQA commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768792633 **[Test build #134584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134584/testReport)** for PR 31373 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31354: [SPARK-34256][ML] VectorSlicer refine numFeatures checking and toString method

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31354: URL: https://github.com/apache/spark/pull/31354#issuecomment-768792498 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39163/

[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768792499 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39165/

[GitHub] [spark] AmplabJenkins commented on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768792501 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39162/

[GitHub] [spark] AmplabJenkins commented on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-768792500 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39161/

[GitHub] [spark] SparkQA commented on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
SparkQA commented on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768792120 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39167/

[GitHub] [spark] SparkQA commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
SparkQA commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768791003 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39166/

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768790916 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39168/

[GitHub] [spark] gatorsmile commented on pull request #26804: [SPARK-26346][BUILD][SQL] Upgrade Parquet to 1.11.1

2021-01-27 Thread GitBox
gatorsmile commented on pull request #26804: URL: https://github.com/apache/spark/pull/26804#issuecomment-768790620 LGTM The current PR looks good to me. However, based on the pervious experience, Parquet upgrade always causes various issues. We might revert the upgrade at the

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

2021-01-27 Thread GitBox
SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768787981 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39169/

<    1   2   3   4   5   6   7   8   >