[GitHub] [spark] SparkQA commented on pull request #31377: [SPARK-34239][SQL] Unify output of SHOW COLUMNS pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31377: URL: https://github.com/apache/spark/pull/31377#issuecomment-768872426 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39178/ ---

[GitHub] [spark] HeartSaVioR commented on pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

2021-01-27 Thread GitBox
HeartSaVioR commented on pull request #31369: URL: https://github.com/apache/spark/pull/31369#issuecomment-768871997 cc. @tdas @zsxwing @jose-torres @gaborgsomogyi @xuanyuanking to see some chance of more eyes to review. Thi

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768871000 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39185/ -

[GitHub] [spark] SparkQA commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
SparkQA commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768870502 **[Test build #134601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134601/testReport)** for PR 31355 at commit [`8b0eb56`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
viirya commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565882779 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -844,18 +844,41 @@ class SessionCatalog( }

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565881107 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -844,18 +844,41 @@ class SessionCatalog(

[GitHub] [spark] maropu commented on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

2021-01-27 Thread GitBox
maropu commented on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-768868009 cc: @HyukjinKwon @ueshin This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] maropu commented on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

2021-01-27 Thread GitBox
maropu commented on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-76886 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565880329 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -844,18 +844,41 @@ class SessionCatalog(

[GitHub] [spark] SparkQA commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
SparkQA commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768866810 **[Test build #134603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134603/testReport)** for PR 31355 at commit [`60ebadf`](https://github.com

[GitHub] [spark] HeartSaVioR commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
HeartSaVioR commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768866804 NOTE to reviewers: I've changed the location of required number of partitions from Distribution interfaces/implementations to RequiresDistributionAndOrdering, so that Distri

[GitHub] [spark] SparkQA commented on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
SparkQA commented on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768866788 **[Test build #134602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134602/testReport)** for PR 31380 at commit [`d7b3108`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565879134 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -844,18 +844,41 @@ class SessionCatalog(

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565879026 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -845,17 +845,33 @@ class SessionCatalog(

[GitHub] [spark] SparkQA commented on pull request #31379: [SPARK-34262][SQL][3.1] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
SparkQA commented on pull request #31379: URL: https://github.com/apache/spark/pull/31379#issuecomment-768865878 **[Test build #134599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134599/testReport)** for PR 31379 at commit [`fb1d319`](https://github.com

[GitHub] [spark] MaxGekk commented on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
MaxGekk commented on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768865311 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] SparkQA commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
SparkQA commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-768863369 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39182/ -

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-768863296 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39177/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
SparkQA removed a comment on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768861368 **[Test build #134598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134598/testReport)** for PR 31380 at commit [`d7b3108`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768861672 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134598/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768861672 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134598/ -

[GitHub] [spark] SparkQA commented on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
SparkQA commented on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768861657 **[Test build #134598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134598/testReport)** for PR 31380 at commit [`d7b3108`](https://github.co

[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
linhongliu-db commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565873980 ## File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql ## @@ -743,20 +743,20 @@ select * from a left join b on i = x and i =

[GitHub] [spark] SparkQA commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
SparkQA commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768861544 **[Test build #134600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134600/testReport)** for PR 31284 at commit [`32e71ce`](https://github.com

[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
linhongliu-db commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565873980 ## File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql ## @@ -743,20 +743,20 @@ select * from a left join b on i = x and i =

[GitHub] [spark] SparkQA commented on pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
SparkQA commented on pull request #31380: URL: https://github.com/apache/spark/pull/31380#issuecomment-768861368 **[Test build #134598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134598/testReport)** for PR 31380 at commit [`d7b3108`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31375: [SPARK-34275][CORE][SQL][MLLIB][3.0] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31375: URL: https://github.com/apache/spark/pull/31375#issuecomment-768861086 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39180/ -

[GitHub] [spark] SparkQA commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768860987 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39184/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768860308 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134579/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31376: [SPARK-34275][CORE][SQL][MLLIB][2.4] Replaces filter and size with count

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31376: URL: https://github.com/apache/spark/pull/31376#issuecomment-768860312 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39179/

[GitHub] [spark] AmplabJenkins commented on pull request #31376: [SPARK-34275][CORE][SQL][MLLIB][2.4] Replaces filter and size with count

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31376: URL: https://github.com/apache/spark/pull/31376#issuecomment-768860312 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39179/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768860308 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134579/ -

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768859984 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39183/ -

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
dongjoon-hyun edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768858103 I confirmed that this bug still exists in the vectorized reader while MR reader can read this correctly like the test Parquet file in this PR. ```scala scala>

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
dongjoon-hyun edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768858103 I confirmed that this bug still exists in the vectorized reader while MR reader can read this correctly like the test Parquet file in this PR. ```scala scala>

[GitHub] [spark] dongjoon-hyun commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
dongjoon-hyun commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768858103 I confirmed that this still exists in the vectorized reader while MR reader can read this correctly like the test Parquet file in this PR. ```scala scala> sql("set sp

[GitHub] [spark] SparkQA commented on pull request #31377: [SPARK-34239][SQL] Unify output of SHOW COLUMNS pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31377: URL: https://github.com/apache/spark/pull/31377#issuecomment-768857701 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39178/ -

[GitHub] [spark] MaxGekk commented on pull request #31361: [SPARK-34262][SQL] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
MaxGekk commented on pull request #31361: URL: https://github.com/apache/spark/pull/31361#issuecomment-768853023 Here are backports: - `branch-3.0`: https://github.com/apache/spark/pull/31380 - `branch-3.1`: https://github.com/apache/spark/pull/31379 -

[GitHub] [spark] viirya commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
viirya commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565867620 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -844,18 +844,41 @@ class SessionCatalog( }

[GitHub] [spark] MaxGekk opened a new pull request #31380: [SPARK-34262][SQL][3.0] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
MaxGekk opened a new pull request #31380: URL: https://github.com/apache/spark/pull/31380 ### What changes were proposed in this pull request? Invoke `CatalogImpl.refreshTable()` in v1 implementation of the `ALTER TABLE .. SET LOCATION` command to refresh cached table data. ### Wh

[GitHub] [spark] cloud-fan commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565865238 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowTablesSuite.scala ## @@ -19,21 +19,15 @@ package org.apache.spark.s

[GitHub] [spark] SparkQA commented on pull request #31376: [SPARK-34275][CORE][SQL][MLLIB][2.4] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31376: URL: https://github.com/apache/spark/pull/31376#issuecomment-768849559 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39179/ ---

[GitHub] [spark] cloud-fan commented on a change in pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
cloud-fan commented on a change in pull request #31284: URL: https://github.com/apache/spark/pull/31284#discussion_r565864134 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala ## @@ -1196,6 +1196,24 @@ class ParquetIO

[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
SparkQA removed a comment on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768754190 **[Test build #134579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134579/testReport)** for PR 31286 at commit [`d02e32c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

2021-01-27 Thread GitBox
SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768849165 **[Test build #134579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134579/testReport)** for PR 31286 at commit [`d02e32c`](https://github.co

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768845586 > Thank you for making a PR, @razajafri . Could you rebase this PR to the master branch please? I have rebased. PTAL --

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
HeartSaVioR commented on a change in pull request #31355: URL: https://github.com/apache/spark/pull/31355#discussion_r565860036 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/distributions/OrderedDistribution.java ## @@ -32,4 +32,13 @@ * Returns or

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768845356 > @razajafri, do you mind clarifying PR description? For exmaple, I thought you meant writing out to files or somewhere by: > > > Spark should read it as a long but writ

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
HeartSaVioR commented on a change in pull request #31355: URL: https://github.com/apache/spark/pull/31355#discussion_r565860036 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/distributions/OrderedDistribution.java ## @@ -32,4 +32,13 @@ * Returns or

[GitHub] [spark] SparkQA commented on pull request #31376: [SPARK-34275][CORE][SQL][MLLIB][2.4] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31376: URL: https://github.com/apache/spark/pull/31376#issuecomment-768842901 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39179/ -

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768841296 **[Test build #134597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134597/testReport)** for PR 31245 at commit [`5efe7f5`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768838442 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134576/ -

[GitHub] [spark] MaxGekk opened a new pull request #31379: [SPARK-34262][SQL][3.1] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
MaxGekk opened a new pull request #31379: URL: https://github.com/apache/spark/pull/31379 ### What changes were proposed in this pull request? Invoke `CatalogImpl.refreshTable()` in v1 implementation of the `ALTER TABLE .. SET LOCATION` command to refresh cached table data. ### Wh

[GitHub] [spark] SparkQA commented on pull request #31377: [SPARK-34239][SQL] Unify output of SHOW COLUMNS pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31377: URL: https://github.com/apache/spark/pull/31377#issuecomment-768838723 **[Test build #134591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134591/testReport)** for PR 31377 at commit [`81d5a38`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-768838711 **[Test build #134590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134590/testReport)** for PR 31378 at commit [`af15746`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768838617 **[Test build #134594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134594/testReport)** for PR 31373 at commit [`510a505`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768838442 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134576/ -

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768838217 **[Test build #134596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134596/testReport)** for PR 31245 at commit [`01845cf`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
SparkQA removed a comment on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768757370 **[Test build #134576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134576/testReport)** for PR 31365 at commit [`cc4645e`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31365: [SPARK-34260][SQL][3.0] Fix UnresolvedException when creating temp view twice

2021-01-27 Thread GitBox
SparkQA commented on pull request #31365: URL: https://github.com/apache/spark/pull/31365#issuecomment-768837466 **[Test build #134576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134576/testReport)** for PR 31365 at commit [`cc4645e`](https://github.co

[GitHub] [spark] razajafri commented on a change in pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
razajafri commented on a change in pull request #31284: URL: https://github.com/apache/spark/pull/31284#discussion_r565853372 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala ## @@ -1196,6 +1196,24 @@ class ParquetIO

[GitHub] [spark] SparkQA commented on pull request #31376: [SPARK-34275][CORE][SQL][MLLIB][2.4] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31376: URL: https://github.com/apache/spark/pull/31376#issuecomment-768834068 **[Test build #134592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134592/testReport)** for PR 31376 at commit [`165a6e6`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-01-27 Thread GitBox
SparkQA commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-768834114 **[Test build #134595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134595/testReport)** for PR 31349 at commit [`22a8979`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31375: [SPARK-34275][CORE][SQL][MLLIB][3.0] Replaces filter and size with count

2021-01-27 Thread GitBox
SparkQA commented on pull request #31375: URL: https://github.com/apache/spark/pull/31375#issuecomment-768834094 **[Test build #134593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134593/testReport)** for PR 31375 at commit [`06c742f`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
dongjoon-hyun commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768833102 Retest this please This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768832812 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768832809 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39171/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768832811 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768832819 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39174/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768832817 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39173/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

2021-01-27 Thread GitBox
AmplabJenkins removed a comment on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768832807 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134578/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31372: [SPARK-34272][SQL] Pretty SQL should check NonSQLExpression

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31372: URL: https://github.com/apache/spark/pull/31372#issuecomment-768832817 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39173/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768832819 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39174/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768832807 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134578/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768832809 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39171/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768832815 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
AmplabJenkins commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768832811 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] LuciferYang commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768832611 thx @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
dongjoon-hyun commented on a change in pull request #31284: URL: https://github.com/apache/spark/pull/31284#discussion_r565849604 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala ## @@ -1196,6 +1196,24 @@ class Parqu

[GitHub] [spark] maropu commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

2021-01-27 Thread GitBox
maropu commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565849237 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -845,17 +845,33 @@ class SessionCatalog( }

[GitHub] [spark] dongjoon-hyun commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
dongjoon-hyun commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768830750 Thank you for pinging me, @HyukjinKwon . Sure. This is an automated message from the Apache Git Service. T

[GitHub] [spark] HyukjinKwon closed pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
HyukjinKwon closed pull request #31374: URL: https://github.com/apache/spark/pull/31374 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768830550 Merged to master and branch-3.1. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768830169 cc @dongjoon-hyun and @cloud-fan, would you guys mind taking a look for this PR too? this is also related to parquet decimals --

[GitHub] [spark] SparkQA removed a comment on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA removed a comment on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768792633 **[Test build #134584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134584/testReport)** for PR 31373 at commit [`0243627`](https://gi

[GitHub] [spark] AngersZhuuuu opened a new pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly

2021-01-27 Thread GitBox
AngersZh opened a new pull request #31378: URL: https://github.com/apache/spark/pull/31378 ### What changes were proposed in this pull request? Passing around the output attributes should have more benefits like keeping the expr ID unchanged to avoid bugs when we apply more operators

[GitHub] [spark] SparkQA removed a comment on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA removed a comment on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768754138 **[Test build #134578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134578/testReport)** for PR 31341 at commit [`fd4afee`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

2021-01-27 Thread GitBox
SparkQA commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-768829498 **[Test build #134584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134584/testReport)** for PR 31373 at commit [`0243627`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

2021-01-27 Thread GitBox
SparkQA commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768829233 **[Test build #134578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134578/testReport)** for PR 31341 at commit [`fd4afee`](https://github.co

[GitHub] [spark] dongjoon-hyun removed a comment on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
dongjoon-hyun removed a comment on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768828864 Why do we need to backport improvement? This is an automated message from the Apache Git Service.

[GitHub] [spark] dongjoon-hyun commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
dongjoon-hyun commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768828864 Why do we need to backport improvement? This is an automated message from the Apache Git Service. To respo

[GitHub] [spark] AngersZhuuuu opened a new pull request #31377: [SPARK-34239][SQL] Unify output of SHOW COLUMNS pass output attributes properly

2021-01-27 Thread GitBox
AngersZh opened a new pull request #31377: URL: https://github.com/apache/spark/pull/31377 ### What changes were proposed in this pull request? Passing around the output attributes should have more benefits like keeping the expr ID unchanged to avoid bugs when we apply more operators

[GitHub] [spark] LuciferYang commented on pull request #31374: [SPARK-34275][CORE][SQL][MLLIB] Replaces filter and size with count

2021-01-27 Thread GitBox
LuciferYang commented on pull request #31374: URL: https://github.com/apache/spark/pull/31374#issuecomment-768823455 @HyukjinKwon - branch-3.0: https://github.com/apache/spark/pull/31375 - branch-2.4: https://github.com/apache/spark/pull/31376 ---

[GitHub] [spark] beliefer commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
beliefer commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565842071 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTablesSuite.scala ## @@ -34,7 +35,7 @@ trait ShowTablesSuiteBase ext

[GitHub] [spark] SparkQA commented on pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

2021-01-27 Thread GitBox
SparkQA commented on pull request #31356: URL: https://github.com/apache/spark/pull/31356#issuecomment-768823442 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39174/ ---

[GitHub] [spark] beliefer commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
beliefer commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565841306 ## File path: docs/sql-migration-guide.md ## @@ -40,6 +40,10 @@ license: | - In Spark 3.2, script transform default FIELD DELIMIT is `\u0001` for no

[GitHub] [spark] SparkQA commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

2021-01-27 Thread GitBox
SparkQA commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768822993 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39176/ ---

[GitHub] [spark] HyukjinKwon commented on pull request #31361: [SPARK-34262][SQL] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31361: URL: https://github.com/apache/spark/pull/31361#issuecomment-768822901 @MaxGekk it has a conflict. Would you mind opening a backport PR? This is an automated message from the Apac

[GitHub] [spark] beliefer commented on a change in pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

2021-01-27 Thread GitBox
beliefer commented on a change in pull request #31245: URL: https://github.com/apache/spark/pull/31245#discussion_r565841011 ## File path: docs/sql-migration-guide.md ## @@ -40,6 +40,10 @@ license: | - In Spark 3.2, script transform default FIELD DELIMIT is `\u0001` for no

[GitHub] [spark] HyukjinKwon commented on pull request #31361: [SPARK-34262][SQL] Refresh cached data of v1 table in `ALTER TABLE .. SET LOCATION`

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31361: URL: https://github.com/apache/spark/pull/31361#issuecomment-768822701 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, pleas

  1   2   3   4   5   6   7   8   >