[GitHub] [spark] HeartSaVioR commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

HeartSaVioR commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768107101 Actually the proposal is more likely giving data source to force having static number of partitions regardless of output data. I see valid concerns about drawbacks wh

[GitHub] [spark] SparkQA commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

SparkQA commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768099637 **[Test build #134539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134539/testReport)** for PR 31355 at commit [`84108e4`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

AmplabJenkins commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768093676 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39121/ -

[GitHub] [spark] HeartSaVioR commented on pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

HeartSaVioR commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-768096504 cc. @cloud-fan @aokolnychyi Let's discuss in this PR, or please let me know if we prefer to have discussion in dev@ mailing list. --

[GitHub] [spark] HeartSaVioR opened a new pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

HeartSaVioR opened a new pull request #31355: URL: https://github.com/apache/spark/pull/31355 ### What changes were proposed in this pull request? This PR proposes to extend the functionality of requirement for distribution and ordering on V2 write to specify the number of partitioni

[GitHub] [spark] SparkQA commented on pull request #31354: [SPARK-34256][ML] VectorSlicer refine numFeatures checking and toString method

SparkQA commented on pull request #31354: URL: https://github.com/apache/spark/pull/31354#issuecomment-768094325 **[Test build #134538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134538/testReport)** for PR 31354 at commit [`877ffbf`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

AmplabJenkins removed a comment on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768093676 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39121/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

AmplabJenkins removed a comment on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768093675 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134534/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

AmplabJenkins removed a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768093672 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

AmplabJenkins commented on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768093675 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134534/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

AmplabJenkins commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768093673 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on pull request #31332: [SPARK-34231][AVRO][TEST] Make proper use of resource file within AvroSuite test case

HyukjinKwon commented on pull request #31332: URL: https://github.com/apache/spark/pull/31332#issuecomment-768088526 Merged to master, branch-3.1, branch-3.0 and branch-2.4. This is an automated message from the Apache Git Se

[GitHub] [spark] HyukjinKwon closed pull request #31332: [SPARK-34231][AVRO][TEST] Make proper use of resource file within AvroSuite test case

HyukjinKwon closed pull request #31332: URL: https://github.com/apache/spark/pull/31332 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA removed a comment on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

SparkQA removed a comment on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768025224 **[Test build #134534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134534/testReport)** for PR 31314 at commit [`91c2acc`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

SparkQA commented on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768086624 **[Test build #134534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134534/testReport)** for PR 31314 at commit [`91c2acc`](https://github.co

[GitHub] [spark] cloud-fan closed pull request #31350: [SPARK-34251][SQL] Fix table stats calculation by `TRUNCATE TABLE`

cloud-fan closed pull request #31350: URL: https://github.com/apache/spark/pull/31350 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #31350: [SPARK-34251][SQL] Fix table stats calculation by `TRUNCATE TABLE`

cloud-fan commented on pull request #31350: URL: https://github.com/apache/spark/pull/31350#issuecomment-768079786 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] cloud-fan closed pull request #31336: [SPARK-34233][SQL] FIX NPE for char padding in binary comparison

cloud-fan closed pull request #31336: URL: https://github.com/apache/spark/pull/31336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] HyukjinKwon commented on pull request #31223: [SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0

HyukjinKwon commented on pull request #31223: URL: https://github.com/apache/spark/pull/31223#issuecomment-768078745 Thank you @dongjoon-hyun. @Ngone51, can we file a JIRA for Scala 2.13 too so we can track? I think we're fine not to downgrade 2.13 for now since 2.13 is not out yet .. Also

[GitHub] [spark] cloud-fan commented on pull request #31336: [SPARK-34233][SQL] FIX NPE for char padding in binary comparison

cloud-fan commented on pull request #31336: URL: https://github.com/apache/spark/pull/31336#issuecomment-768078650 GA passed, merging to master/3.1! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] zhengruifeng opened a new pull request #31354: [SPARK-34256][ML] VectorSlicer refine numFeatures checking and toString method

zhengruifeng opened a new pull request #31354: URL: https://github.com/apache/spark/pull/31354 ### What changes were proposed in this pull request? 1, update checking of numFeatures; 2, update `toString` to take `names` into account; ### Why are the changes needed? 1, sho

[GitHub] [spark] razajafri edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768074328 > The code looks fine to me but do you have any tests to show what the performance impact might be to other parquet files that have decimal values written as they typic

[GitHub] [spark] dongjoon-hyun commented on pull request #31223: [SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0

dongjoon-hyun commented on pull request #31223: URL: https://github.com/apache/spark/pull/31223#issuecomment-768074365 To be clear, I'm fine with reverting PR here. This is an automated message from the Apache Git Service. To

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768074328 > The code looks fine to me but do you have any tests to show what the performance impact might be to other parquet files that have decimal values written as they typically ar

[GitHub] [spark] dongjoon-hyun commented on pull request #31223: [SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0

dongjoon-hyun commented on pull request #31223: URL: https://github.com/apache/spark/pull/31223#issuecomment-768073802 BTW, are we going to downgrade Scala 2.13, too? It seems that the scala issue is open for 2.13, too. This

[GitHub] [spark] razajafri edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768066093 > I think this can be revised a bit to make it more understandable. I guess another approach is to initialize the `WriteColumnVector` to use `long` array instead of `in

[GitHub] [spark] SparkQA commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

SparkQA commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768072620 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39121/ ---

[GitHub] [spark] dongjoon-hyun commented on pull request #31223: [SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0

dongjoon-hyun commented on pull request #31223: URL: https://github.com/apache/spark/pull/31223#issuecomment-768072299 Thank you for reporting. Do you think we can add a test case for that with or after reverting? Thi

[GitHub] [spark] Ngone51 commented on pull request #31223: [SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0

Ngone51 commented on pull request #31223: URL: https://github.com/apache/spark/pull/31223#issuecomment-768066319 Hi all, I find there's still a regression of scala as reported in https://github.com/scala/bug/issues/12038. I tested the demo locally and verified that the regression does exis

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768066093 > I think this can be revised a bit to make it more understandable. I guess another approach is to initialize the `WriteColumnVector` to use `long` array instead of `int`. It

[GitHub] [spark] viirya commented on a change in pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on a change in pull request #31296: URL: https://github.com/apache/spark/pull/31296#discussion_r565055386 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ## @@ -2007,6 +2007,54 @@ class DatasetSuite extends QueryTest checkAn

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768065179 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134529/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

AmplabJenkins removed a comment on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768065180 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39119/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

AmplabJenkins removed a comment on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768065181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39117/

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768065179 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134529/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

AmplabJenkins commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768065181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39117/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

AmplabJenkins commented on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768065180 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39119/ -

[GitHub] [spark] viirya commented on a change in pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on a change in pull request #31296: URL: https://github.com/apache/spark/pull/31296#discussion_r565055386 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ## @@ -2007,6 +2007,54 @@ class DatasetSuite extends QueryTest checkAn

[GitHub] [spark] viirya commented on a change in pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on a change in pull request #31296: URL: https://github.com/apache/spark/pull/31296#discussion_r565055386 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ## @@ -2007,6 +2007,54 @@ class DatasetSuite extends QueryTest checkAn

[GitHub] [spark] viirya commented on a change in pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on a change in pull request #31296: URL: https://github.com/apache/spark/pull/31296#discussion_r565055386 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ## @@ -2007,6 +2007,54 @@ class DatasetSuite extends QueryTest checkAn

[GitHub] [spark] SparkQA commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

SparkQA commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768060093 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39121/ -

[GitHub] [spark] viirya commented on a change in pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

viirya commented on a change in pull request #31207: URL: https://github.com/apache/spark/pull/31207#discussion_r565052739 ## File path: python/pyspark/sql/functions.py ## @@ -91,13 +92,48 @@ def lit(col): Creates a :class:`Column` of literal value. .. versionadded:

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

AngersZh commented on a change in pull request #31296: URL: https://github.com/apache/spark/pull/31296#discussion_r565052466 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ## @@ -2007,6 +2007,54 @@ class DatasetSuite extends QueryTest c

[GitHub] [spark] LuciferYang edited a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

LuciferYang edited a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-768056895 > Could you file the corresponding JIRA to Apache Parquet and Apache ORC community and link them in the JIRA and the PR description? [PARQUET-1965](https://issu

[GitHub] [spark] LuciferYang commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

LuciferYang commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-768056895 > Could you file the corresponding JIRA to Apache Parquet and Apache ORC community and link them in the JIRA and the PR description? PARQUET-1965 and ORC-746 ---

[GitHub] [spark] SparkQA removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-767950588 **[Test build #134529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134529/testReport)** for PR 31245 at commit [`6b15233`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768053283 **[Test build #134529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134529/testReport)** for PR 31245 at commit [`6b15233`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

SparkQA commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768051051 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39117/ ---

[GitHub] [spark] cloud-fan commented on pull request #30829: [SPARK-33832][SQL] Add an option in AQE to mitigate skew even if it c…

cloud-fan commented on pull request #30829: URL: https://github.com/apache/spark/pull/30829#issuecomment-768047646 Ah sorry for my bad memory, so the comment in `OptimizeSkewedJoin` is actually the stale one... I get your point that it's like backtracking, but it does make the contro

[GitHub] [spark] SparkQA commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

SparkQA commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768045693 **[Test build #134536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134536/testReport)** for PR 31352 at commit [`7016fa5`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768024811 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31336: [SPARK-34233][SQL] FIX NPE for char padding in binary comparison

AmplabJenkins removed a comment on pull request #31336: URL: https://github.com/apache/spark/pull/31336#issuecomment-768041718 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39118/

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768041717 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134531/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31336: [SPARK-34233][SQL] FIX NPE for char padding in binary comparison

AmplabJenkins commented on pull request #31336: URL: https://github.com/apache/spark/pull/31336#issuecomment-768041718 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39118/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

AmplabJenkins removed a comment on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768041107 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39120/

[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

AmplabJenkins commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768041107 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39120/ -

[GitHub] [spark] SparkQA commented on pull request #31336: [SPARK-34233][SQL] FIX NPE for char padding in binary comparison

SparkQA commented on pull request #31336: URL: https://github.com/apache/spark/pull/31336#issuecomment-768040115 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39118/ -

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768037994 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39120/ ---

[GitHub] [spark] SparkQA commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

SparkQA commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768037242 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39117/ -

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768036417 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39120/ -

[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

cloud-fan commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565032049 ## File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql ## @@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y a

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon commented on a change in pull request #31207: URL: https://github.com/apache/spark/pull/31207#discussion_r565030565 ## File path: python/pyspark/sql/functions.py ## @@ -91,13 +92,48 @@ def lit(col): Creates a :class:`Column` of literal value. .. versiona

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon commented on a change in pull request #31207: URL: https://github.com/apache/spark/pull/31207#discussion_r565030192 ## File path: python/pyspark/sql/functions.py ## @@ -91,13 +92,48 @@ def lit(col): Creates a :class:`Column` of literal value. .. versiona

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon commented on a change in pull request #31207: URL: https://github.com/apache/spark/pull/31207#discussion_r565027496 ## File path: python/pyspark/sql/functions.py ## @@ -91,13 +92,48 @@ def lit(col): Creates a :class:`Column` of literal value. .. versiona

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon commented on a change in pull request #31207: URL: https://github.com/apache/spark/pull/31207#discussion_r565027496 ## File path: python/pyspark/sql/functions.py ## @@ -91,13 +92,48 @@ def lit(col): Creates a :class:`Column` of literal value. .. versiona

[GitHub] [spark] imback82 commented on a change in pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

imback82 commented on a change in pull request #31352: URL: https://github.com/apache/spark/pull/31352#discussion_r565027977 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala ## @@ -53,38 +53,42 @@ import org.apache.spark.sql.catalyst.

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon commented on a change in pull request #31207: URL: https://github.com/apache/spark/pull/31207#discussion_r565027496 ## File path: python/pyspark/sql/functions.py ## @@ -91,13 +92,48 @@ def lit(col): Creates a :class:`Column` of literal value. .. versiona

[GitHub] [spark] HyukjinKwon removed a comment on pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon removed a comment on pull request #31207: URL: https://github.com/apache/spark/pull/31207#issuecomment-768022562 Oops, I missed this Pr. Yeah, make sense. This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

SparkQA commented on pull request #31314: URL: https://github.com/apache/spark/pull/31314#issuecomment-768025224 **[Test build #134534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134534/testReport)** for PR 31314 at commit [`91c2acc`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768024811 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39116/ -

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768024727 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39116/ ---

[GitHub] [spark] imback82 commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

imback82 commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768023838 > Let's remove the regression mentioned in PR description. Seems like it's not a regression. In Spark 3.1.1-RC, if we start `spark-shell` with the `SPARK_TESTING=true` e

[GitHub] [spark] akiyamaneko commented on a change in pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

akiyamaneko commented on a change in pull request #31314: URL: https://github.com/apache/spark/pull/31314#discussion_r565022689 ## File path: core/src/main/resources/org/apache/spark/ui/static/stagepage.js ## @@ -882,7 +882,9 @@ $(document).ready(function () {

[GitHub] [spark] HyukjinKwon commented on pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

HyukjinKwon commented on pull request #31207: URL: https://github.com/apache/spark/pull/31207#issuecomment-768022562 Oops, I missed this Pr. Yeah, make sense. This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768021322 **[Test build #134535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134535/testReport)** for PR 31286 at commit [`30aa4db`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31336: [SPARK-34233][SQL] FIX NPE for char padding in binary comparison

SparkQA commented on pull request #31336: URL: https://github.com/apache/spark/pull/31336#issuecomment-768021278 **[Test build #134533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134533/testReport)** for PR 31336 at commit [`67e06fa`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

SparkQA commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-768021257 **[Test build #134532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134532/testReport)** for PR 31341 at commit [`9975150`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768020135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39114/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

AmplabJenkins removed a comment on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768020137 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

AmplabJenkins commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768020137 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768020135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39114/ -

[GitHub] [spark] cloud-fan commented on a change in pull request #31319: [SPARK-34212][SQL] Fix incorrect decimal reading from Parquet files

cloud-fan commented on a change in pull request #31319: URL: https://github.com/apache/spark/pull/31319#discussion_r565018819 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -3868,6 +3869,57 @@ class SQLQuerySuite extends QueryTest with Sha

[GitHub] [spark] HyukjinKwon commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

HyukjinKwon commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768018643 Let's remove the regression mentioned in PR description. Seems like it's not a regression. This is an autom

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

AngersZh commented on a change in pull request #31341: URL: https://github.com/apache/spark/pull/31341#discussion_r565018727 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowPartitionsSuite.scala ## @@ -126,4 +127,22 @@ class ShowPartitio

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768018441 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39116/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA removed a comment on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-767973237 **[Test build #134530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134530/testReport)** for PR 31286 at commit [`ba80913`](https://gi

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

HyukjinKwon commented on a change in pull request #31352: URL: https://github.com/apache/spark/pull/31352#discussion_r565018271 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala ## @@ -53,38 +53,42 @@ import org.apache.spark.sql.cataly

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768018132 **[Test build #134530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134530/testReport)** for PR 31286 at commit [`ba80913`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #31319: [SPARK-34212][SQL] Fix incorrect decimal reading from Parquet files

cloud-fan commented on a change in pull request #31319: URL: https://github.com/apache/spark/pull/31319#discussion_r565018052 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -3868,6 +3869,57 @@ class SQLQuerySuite extends QueryTest with Sha

[GitHub] [spark] sarutak commented on a change in pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

sarutak commented on a change in pull request #31314: URL: https://github.com/apache/spark/pull/31314#discussion_r565017747 ## File path: core/src/main/resources/org/apache/spark/ui/static/stagepage.js ## @@ -882,7 +882,9 @@ $(document).ready(function () {

[GitHub] [spark] HyukjinKwon commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

HyukjinKwon commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-768017199 cc @jiangxb1987 would you mind taking a look and merging this one when you find some time please? This is a

[GitHub] [spark] sarutak commented on a change in pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

sarutak commented on a change in pull request #31314: URL: https://github.com/apache/spark/pull/31314#discussion_r565016978 ## File path: core/src/main/resources/org/apache/spark/ui/static/stagepage.js ## @@ -882,7 +882,9 @@ $(document).ready(function () {

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31254: [SPARK-34165][SQL] Add count_distinct as an option to Dataset#summary

HyukjinKwon commented on a change in pull request #31254: URL: https://github.com/apache/spark/pull/31254#discussion_r565016159 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -2670,6 +2670,7 @@ class Dataset[T] private[sql]( * min * m

[GitHub] [spark] akiyamaneko commented on a change in pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

akiyamaneko commented on a change in pull request #31314: URL: https://github.com/apache/spark/pull/31314#discussion_r565015088 ## File path: core/src/main/resources/org/apache/spark/ui/static/stagepage.js ## @@ -882,7 +882,9 @@ $(document).ready(function () {

[GitHub] [spark] cloud-fan closed pull request #31339: [SPARK-34236][SQL] Fix v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

cloud-fan closed pull request #31339: URL: https://github.com/apache/spark/pull/31339 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #31339: [SPARK-34236][SQL] Fix v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

cloud-fan commented on pull request #31339: URL: https://github.com/apache/spark/pull/31339#issuecomment-768008606 I'm merging in to unblock the other bug fix https://github.com/apache/spark/pull/31336 , reviewers please keep leaving comments if you have any. Thanks, merging to mast

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-768007726 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39115/ ---

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-768005929 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39114/ ---

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-76847 **[Test build #134531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134531/testReport)** for PR 31245 at commit [`148127e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

SparkQA commented on pull request #31286: URL: https://github.com/apache/spark/pull/31286#issuecomment-767994552 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39115/ -

  1   2   3   4   5   6   7   >