[GitHub] [spark] SparkQA commented on pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

SparkQA commented on pull request #33532: URL: https://github.com/apache/spark/pull/33532#issuecomment-887264795 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46198/ -- This is an automated message from the A

[GitHub] [spark] Ngone51 commented on a change in pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

Ngone51 commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r677174779 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -725,15 +896,32 @@ public void onD

[GitHub] [spark] SparkQA commented on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

SparkQA commented on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887265668 **[Test build #141680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141680/testReport)** for PR 33531 at commit [`d79bdbf`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887266521 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46202/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887268062 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46201/ -- This is an automated message from the Apache

[GitHub] [spark] zero323 closed pull request #33399: [SPARK-36211][PYTHON] Correct typing of `udf` return value

zero323 closed pull request #33399: URL: https://github.com/apache/spark/pull/33399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] Ngone51 commented on a change in pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

Ngone51 commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r677178309 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -668,9 +839,9 @@ public void onDat

[GitHub] [spark] LuciferYang commented on pull request #33350: [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

LuciferYang commented on pull request #33350: URL: https://github.com/apache/spark/pull/33350#issuecomment-887269393 @sunchao I found that the old case use `spark.master local[1]`, but `SharedSparkSession` create `TestSparkSession` with `local[2]` as default , so we should override `cr

[GitHub] [spark] LuciferYang edited a comment on pull request #33350: [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

LuciferYang edited a comment on pull request #33350: URL: https://github.com/apache/spark/pull/33350#issuecomment-887269393 @sunchao I found that the old case use `spark.master local[1]`, but `SharedSparkSession` create `TestSparkSession` with `local[2]` as default , so we should overr

[GitHub] [spark] SparkQA commented on pull request #33533: Revert "[SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

SparkQA commented on pull request #33533: URL: https://github.com/apache/spark/pull/33533#issuecomment-887269812 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46200/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

SparkQA commented on pull request #33532: URL: https://github.com/apache/spark/pull/33532#issuecomment-887269974 **[Test build #141684 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141684/testReport)** for PR 33532 at commit [`d83429e`](https://github.co

[GitHub] [spark] zero323 commented on pull request #33399: [SPARK-36211][PYTHON] Correct typing of `udf` return value

zero323 commented on pull request #33399: URL: https://github.com/apache/spark/pull/33399#issuecomment-887270160 Merged to master, branch-3.2 and branch-3.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] SparkQA removed a comment on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

SparkQA removed a comment on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887197148 **[Test build #141680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141680/testReport)** for PR 33531 at commit [`d79bdbf`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

SparkQA removed a comment on pull request #33532: URL: https://github.com/apache/spark/pull/33532#issuecomment-887218117 **[Test build #141684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141684/testReport)** for PR 33532 at commit [`d83429e`](https://gi

[GitHub] [spark] MaxGekk commented on a change in pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

MaxGekk commented on a change in pull request #33518: URL: https://github.com/apache/spark/pull/33518#discussion_r677182427 ## File path: docs/sql-ref-datatypes.md ## @@ -49,6 +49,37 @@ Spark SQL and DataFrames support the following data types: absolute point in time. - `

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AngersZh commented on a change in pull request #33518: URL: https://github.com/apache/spark/pull/33518#discussion_r677183164 ## File path: docs/sql-ref-datatypes.md ## @@ -49,6 +49,37 @@ Spark SQL and DataFrames support the following data types: absolute point in time.

[GitHub] [spark] AmplabJenkins commented on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

AmplabJenkins commented on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887274956 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141680/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33533: Revert "[SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

AmplabJenkins commented on pull request #33533: URL: https://github.com/apache/spark/pull/33533#issuecomment-887274954 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46200/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

AmplabJenkins removed a comment on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887253143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33533: Revert "[SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

AmplabJenkins removed a comment on pull request #33533: URL: https://github.com/apache/spark/pull/33533#issuecomment-887274954 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46200/

[GitHub] [spark] AmplabJenkins commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

AmplabJenkins commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887274950 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141682/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

AmplabJenkins commented on pull request #33532: URL: https://github.com/apache/spark/pull/33532#issuecomment-887274953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

AmplabJenkins removed a comment on pull request #33532: URL: https://github.com/apache/spark/pull/33532#issuecomment-887274951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

AmplabJenkins removed a comment on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887274950 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141682/ -

[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-887275607 **[Test build #141689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141689/testReport)** for PR 33200 at commit [`8dcc44d`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887277494 **[Test build #141690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141690/testReport)** for PR 33518 at commit [`f0a2164`](https://github.com

[GitHub] [spark] Ngone51 commented on pull request #33514: [SPARK-36242][CORE][3.0] Ensure spill file closed before set success = true in ExternalSorter.spillMemoryIteratorToDisk method

Ngone51 commented on pull request #33514: URL: https://github.com/apache/spark/pull/33514#issuecomment-887283200 > Curious why the earlier PR could not have been merged to 3.1/3.0 @mridulm My bad. I didn't merge it since I thought it was an improvement. -- This is an automated mess

[GitHub] [spark] cloud-fan commented on a change in pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

cloud-fan commented on a change in pull request #33518: URL: https://github.com/apache/spark/pull/33518#discussion_r677196111 ## File path: docs/sql-ref-datatypes.md ## @@ -49,6 +49,44 @@ Spark SQL and DataFrames support the following data types: absolute point in time. -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887285840 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141690/ -

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887285607 **[Test build #141690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141690/testReport)** for PR 33518 at commit [`f0a2164`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887277494 **[Test build #141690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141690/testReport)** for PR 33518 at commit [`f0a2164`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887285840 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141690/ -- This

[GitHub] [spark] MaxGekk commented on a change in pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

MaxGekk commented on a change in pull request #33518: URL: https://github.com/apache/spark/pull/33518#discussion_r677198955 ## File path: docs/sql-ref-datatypes.md ## @@ -49,6 +49,44 @@ Spark SQL and DataFrames support the following data types: absolute point in time. - `

[GitHub] [spark] LuciferYang commented on pull request #33350: [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

LuciferYang commented on pull request #33350: URL: https://github.com/apache/spark/pull/33350#issuecomment-887291000 It seems that `SPARK-36128: spark.sql.hive.metastorePartitionPruning should work for file data sources` should not be placed in sql/core module. -- This is an automated me

[GitHub] [spark] HyukjinKwon commented on pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

HyukjinKwon commented on pull request #33532: URL: https://github.com/apache/spark/pull/33532#issuecomment-887291133 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon closed pull request #33532: [SPARK-36285][INFRA][TESTS] Skip MiMa in PySpark/SparkR/Docker GHA job

HyukjinKwon closed pull request #33532: URL: https://github.com/apache/spark/pull/33532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-887293304 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46203/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33441: [SPARK-33865][SPARK-36202][SQL] When HiveDDL, we need check avro schema too

SparkQA commented on pull request #33441: URL: https://github.com/apache/spark/pull/33441#issuecomment-887294728 **[Test build #141678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141678/testReport)** for PR 33441 at commit [`5b99ab6`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33441: [SPARK-33865][SPARK-36202][SQL] When HiveDDL, we need check avro schema too

SparkQA removed a comment on pull request #33441: URL: https://github.com/apache/spark/pull/33441#issuecomment-887180308 **[Test build #141678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141678/testReport)** for PR 33441 at commit [`5b99ab6`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887297032 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46202/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887299273 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46201/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang commented on pull request #33514: [SPARK-36242][CORE][3.0] Ensure spill file closed before set success = true in ExternalSorter.spillMemoryIteratorToDisk method

LuciferYang commented on pull request #33514: URL: https://github.com/apache/spark/pull/33514#issuecomment-887299851 > My bad. I didn't merge it since I thought it was an improvement. Sorry, It's my description in jira that makes @Ngone51 misunderstood, before completed the new test

[GitHub] [spark] shardulm94 commented on pull request #33446: [SPARK-36215][SHUFFLE] Add logging for slow fetches to diagnose external shuffle service issues

shardulm94 commented on pull request #33446: URL: https://github.com/apache/spark/pull/33446#issuecomment-887304237 I am not completely sure how showing the 5%ile would work. 1) We will need a reasonable number of shuffles on an executor before we can even calculate 5%ile. We cannot dedu

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887311025 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46204/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on pull request #33525: [SPARK-35320][SQL] Improve error message for unsupported key types in MapType in from_json expression

cloud-fan commented on pull request #33525: URL: https://github.com/apache/spark/pull/33525#issuecomment-887312158 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887312795 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46202/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33441: [SPARK-33865][SPARK-36202][SQL] When HiveDDL, we need check avro schema too

AmplabJenkins commented on pull request #33441: URL: https://github.com/apache/spark/pull/33441#issuecomment-887312796 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141678/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887312795 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46202/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887312798 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46201/

[GitHub] [spark] AmplabJenkins commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887312798 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46201/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33441: [SPARK-33865][SPARK-36202][SQL] When HiveDDL, we need check avro schema too

AmplabJenkins removed a comment on pull request #33441: URL: https://github.com/apache/spark/pull/33441#issuecomment-887312796 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141678/ -

[GitHub] [spark] cloud-fan commented on a change in pull request #33525: [SPARK-35320][SQL] Improve error message for unsupported key types in MapType in from_json expression

cloud-fan commented on a change in pull request #33525: URL: https://github.com/apache/spark/pull/33525#discussion_r677228668 ## File path: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala ## @@ -390,11 +390,15 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33525: [SPARK-35320][SQL] Improve error message for unsupported key types in MapType in from_json expression

AmplabJenkins removed a comment on pull request #33525: URL: https://github.com/apache/spark/pull/33525#issuecomment-886962058 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] SparkQA commented on pull request #33525: [SPARK-35320][SQL] Improve error message for unsupported key types in MapType in from_json expression

SparkQA commented on pull request #33525: URL: https://github.com/apache/spark/pull/33525#issuecomment-887314862 **[Test build #141691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141691/testReport)** for PR 33525 at commit [`b2ed6b9`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887314844 **[Test build #141692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141692/testReport)** for PR 33518 at commit [`0b22603`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-887320944 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46203/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

AmplabJenkins commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-887320971 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46203/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

AmplabJenkins removed a comment on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-887320971 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46203/

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887325505 **[Test build #141692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141692/testReport)** for PR 33518 at commit [`0b22603`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887325782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141692/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887325782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141692/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887314844 **[Test build #141692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141692/testReport)** for PR 33518 at commit [`0b22603`](https://gi

[GitHub] [spark] dgd-contributor opened a new pull request #33534: [SPARK-36099][CORE] Grouping exception in core/util

dgd-contributor opened a new pull request #33534: URL: https://github.com/apache/spark/pull/33534 ### What changes were proposed in this pull request? This PR group exception messages in core/src/main/scala/org/apache/spark/util ### Why are the changes needed? It will largely he

[GitHub] [spark] AngersZhuuuu commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AngersZh commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887328231 > Shall we make the RESTFUL request hang and the web page loading if the spark application is not fully started? Show as below is ok? ![image](https://user-images

[GitHub] [spark] AngersZhuuuu commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AngersZh commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887328461 > > With this 500 and error stack in the log makes user confused too.. they always ask me if there is something wong. > > At least before the changes it shows hint "i

[GitHub] [spark] beliefer opened a new pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

beliefer opened a new pull request #33535: URL: https://github.com/apache/spark/pull/33535 ### What changes were proposed in this pull request? This PR refactor some exceptions in `QueryParsingErrors` to use error classes. There are currently ~100 exceptions in this file; so this

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887331497 **[Test build #141688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141688/testReport)** for PR 31517 at commit [`33f5353`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887245593 **[Test build #141688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141688/testReport)** for PR 31517 at commit [`33f5353`](https://gi

[GitHub] [spark] LuciferYang commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

LuciferYang commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887333011 waiting https://github.com/apache/spark/pull/33533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887336479 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46204/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

SparkQA commented on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887336694 **[Test build #141685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141685/testReport)** for PR 33531 at commit [`eed72e1`](https://github.co

[GitHub] [spark] dgd-contributor opened a new pull request #33536: [SPARK-36101][CORE] Grouping exception in core/api

dgd-contributor opened a new pull request #33536: URL: https://github.com/apache/spark/pull/33536 ### What changes were proposed in this pull request? This PR group exception messages in core/src/main/scala/org/apache/spark/api ### Why are the changes needed? It will largely hel

[GitHub] [spark] SparkQA commented on pull request #33525: [SPARK-35320][SQL] Improve error message for unsupported key types in MapType in from_json expression

SparkQA commented on pull request #33525: URL: https://github.com/apache/spark/pull/33525#issuecomment-887342935 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46205/ -- This is an automated message from the Apache

[GitHub] [spark] yutoacts opened a new pull request #33537: [SPARK-595][DOCS] Add local-cluster mode option in Documentation

yutoacts opened a new pull request #33537: URL: https://github.com/apache/spark/pull/33537 ### What changes were proposed in this pull request? Add local-cluster mode option to submitting-applications.md ### Why are the changes needed? Help users to find/use this

[GitHub] [spark] SparkQA removed a comment on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

SparkQA removed a comment on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887218140 **[Test build #141685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141685/testReport)** for PR 33531 at commit [`eed72e1`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

AmplabJenkins commented on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887353778 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141685/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887353779 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46204/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887353780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141688/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

AmplabJenkins removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887353779 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46204/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-887353780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141688/ -

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887354587 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46206/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

AmplabJenkins removed a comment on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-887353778 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141685/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33534: [SPARK-36099][CORE] Grouping exception in core/util

AmplabJenkins commented on pull request #33534: URL: https://github.com/apache/spark/pull/33534#issuecomment-887354962 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #33536: [SPARK-36101][CORE] Grouping exception in core/api

AmplabJenkins commented on pull request #33536: URL: https://github.com/apache/spark/pull/33536#issuecomment-887354887 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #33537: [SPARK-595][DOCS] Add local-cluster mode option in Documentation

AmplabJenkins commented on pull request #33537: URL: https://github.com/apache/spark/pull/33537#issuecomment-887354840 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

SparkQA commented on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-887356563 **[Test build #141693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141693/testReport)** for PR 33535 at commit [`32b28e0`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

SparkQA commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887356780 **[Test build #141694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141694/testReport)** for PR 33457 at commit [`dba26cd`](https://github.com

[GitHub] [spark] cloud-fan commented on pull request #33488: [SPARK-36241][SQL] Support creating tables with null column

cloud-fan commented on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-887361488 thanks, merging to master/3.2! (since it removes the constraint added in 3.2) -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] cloud-fan closed pull request #33488: [SPARK-36241][SQL] Support creating tables with null column

cloud-fan closed pull request #33488: URL: https://github.com/apache/spark/pull/33488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] Peng-Lei opened a new pull request #33538: [WIP][SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei opened a new pull request #33538: URL: https://github.com/apache/spark/pull/33538 ### What changes were proposed in this pull request? Refactor some exceptions in QueryExecutionErrors to use error classes. as follows: ``` columnChangeUnsupportedError logicalHintOperator

[GitHub] [spark] AmplabJenkins commented on pull request #33538: [WIP][SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

AmplabJenkins commented on pull request #33538: URL: https://github.com/apache/spark/pull/33538#issuecomment-887364962 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #33525: [SPARK-35320][SQL] Improve error message for unsupported key types in MapType in from_json expression

SparkQA commented on pull request #33525: URL: https://github.com/apache/spark/pull/33525#issuecomment-887367749 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46205/ -- This is an automated message from the A

[GitHub] [spark] yutoacts commented on pull request #33537: [SPARK-595][DOCS] Add local-cluster mode option in Documentation

yutoacts commented on pull request #33537: URL: https://github.com/apache/spark/pull/33537#issuecomment-887373601 https://issues.apache.org/jira/browse/SPARK-595 This issue has not been solved and ended up closed although people on comments seem to conclude that the documentation should

[GitHub] [spark] sammyjmoseley commented on a change in pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

sammyjmoseley commented on a change in pull request #33364: URL: https://github.com/apache/spark/pull/33364#discussion_r677296396 ## File path: python/pyspark/sql/dataframe.py ## @@ -1980,6 +1980,9 @@ def dropDuplicates(self, subset=None): |Alice| 5|80| +

[GitHub] [spark] yutoacts edited a comment on pull request #33537: [SPARK-595][DOCS] Add local-cluster mode option in Documentation

yutoacts edited a comment on pull request #33537: URL: https://github.com/apache/spark/pull/33537#issuecomment-887373601 https://issues.apache.org/jira/browse/SPARK-595?focusedCommentId=14292309&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14292309 This i

[GitHub] [spark] SparkQA commented on pull request #33533: Revert "[SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

SparkQA commented on pull request #33533: URL: https://github.com/apache/spark/pull/33533#issuecomment-887376720 **[Test build #141686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141686/testReport)** for PR 33533 at commit [`a733179`](https://github.co

[GitHub] [spark] sarutak closed pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

sarutak closed pull request #33518: URL: https://github.com/apache/spark/pull/33518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] sarutak commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

sarutak commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-887383934 Merged to `master` and `branch-3.2`. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

HyukjinKwon commented on a change in pull request #33364: URL: https://github.com/apache/spark/pull/33364#discussion_r677306998 ## File path: python/pyspark/sql/dataframe.py ## @@ -1980,6 +1980,9 @@ def dropDuplicates(self, subset=None): |Alice| 5|80| +--

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

HyukjinKwon commented on a change in pull request #33364: URL: https://github.com/apache/spark/pull/33364#discussion_r677307453 ## File path: python/pyspark/sql/tests/test_dataframe.py ## @@ -67,6 +67,32 @@ def test_help_command(self): pydoc.render_doc(df.foo)

  1   2   3   4   5   6   7   8   9   >