[GitHub] [spark] AmplabJenkins removed a comment on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

AmplabJenkins removed a comment on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771312661 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134745/

[GitHub] [spark] AmplabJenkins commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

AmplabJenkins commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771312661 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134745/

[GitHub] [spark] AmplabJenkins commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771312662 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134746/

[GitHub] [spark] LuciferYang edited a comment on pull request #31416: [SPARK-34310][CORE][SQL] Replaces map and flatten with flatMap

LuciferYang edited a comment on pull request #31416: URL: https://github.com/apache/spark/pull/31416#issuecomment-771310680 thx @srowen @HyukjinKwon This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] LuciferYang commented on pull request #31416: [SPARK-34310][CORE][SQL] Replaces map and flatten with flatMap

LuciferYang commented on pull request #31416: URL: https://github.com/apache/spark/pull/31416#issuecomment-771310680 thx @dongjoon-hyun @HyukjinKwon This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] dongjoon-hyun commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.0

dongjoon-hyun commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-771308098 Thank you for keeping working on this area. This is an automated message from the Apache Git Service. To

[GitHub] [spark] beliefer commented on a change in pull request #31356: [SPARK-34257][SQL] Improve performance for last_value over unbounded window frame

beliefer commented on a change in pull request #31356: URL: https://github.com/apache/spark/pull/31356#discussion_r568282572 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala ## @@ -500,8 +501,18 @@ final class

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31428: [SPARK-34316][K8S] Support spark.kubernetes.executor.disableConfigMap

dongjoon-hyun edited a comment on pull request #31428: URL: https://github.com/apache/spark/pull/31428#issuecomment-771306218 Could you review this, @ScrapCodes ? There was a user request to disable the new feature of SPARK-30985 for some security reasons.

[GitHub] [spark] dongjoon-hyun commented on pull request #31428: [SPARK-34316][K8S] Support spark.kubernetes.executor.disableConfigMap

dongjoon-hyun commented on pull request #31428: URL: https://github.com/apache/spark/pull/31428#issuecomment-771306218 Could you review this, @ScrapCodes ? This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun opened a new pull request #31428: [SPARK-34316][K8S] Support spark.kubernetes.executor.disableConfigMap

dongjoon-hyun opened a new pull request #31428: URL: https://github.com/apache/spark/pull/31428 ### What changes were proposed in this pull request? This PR aims to add a new configuration `spark.kubernetes.executor.disableConfigMap`. ### Why are the changes needed?

[GitHub] [spark] SparkQA removed a comment on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

SparkQA removed a comment on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771189398 **[Test build #134745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134745/testReport)** for PR 31424 at commit

[GitHub] [spark] SparkQA commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

SparkQA commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771303573 **[Test build #134745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134745/testReport)** for PR 31424 at commit

[GitHub] [spark] beliefer commented on a change in pull request #31316: [SPARK-33599][SQL][FOLLOWUP] Group exception messages in catalyst/analysis

beliefer commented on a change in pull request #31316: URL: https://github.com/apache/spark/pull/31316#discussion_r568278604 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala ## @@ -732,4 +732,20 @@ private[spark] object

[GitHub] [spark] SparkQA removed a comment on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA removed a comment on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771186832 **[Test build #134746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134746/testReport)** for PR 31413 at commit

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771299842 **[Test build #134746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134746/testReport)** for PR 31413 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31426: [SPARK-34318][SQL] Dataset.colRegex should work with column names and qualifiers which contain newlines

AmplabJenkins commented on pull request #31426: URL: https://github.com/apache/spark/pull/31426#issuecomment-771288052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39333/

[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

maropu commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r568261101 ## File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql ## @@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and

[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

maropu commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r568260449 ## File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql ## @@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and

[GitHub] [spark] imback82 commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

imback82 commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771279569 cc @cloud-fan @MaxGekk This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] maropu commented on a change in pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

maropu commented on a change in pull request #31413: URL: https://github.com/apache/spark/pull/31413#discussion_r568257272 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -591,20 +590,48 @@ case class FileSourceScanExec(

[GitHub] [spark] sunchao commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.0

sunchao commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-771277246 This is staled because Spark can't use Hadoop 3.3.0 as it is (see HADOOP-16080). The work may be resumed once Hadoop 3.3.1 is released.

[GitHub] [spark] SparkQA commented on pull request #31426: [SPARK-34318][SQL] Dataset.colRegex should work with column names and qualifiers which contain newlines

SparkQA commented on pull request #31426: URL: https://github.com/apache/spark/pull/31426#issuecomment-771276922 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39333/

[GitHub] [spark] sunchao commented on a change in pull request #29542: [SPARK-32703][SQL] Replace deprecated API calls from SpecificParquetRecordReaderBase

sunchao commented on a change in pull request #29542: URL: https://github.com/apache/spark/pull/29542#discussion_r568255088 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java ## @@ -92,67 +88,23 @@

[GitHub] [spark] maropu commented on a change in pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

maropu commented on a change in pull request #31413: URL: https://github.com/apache/spark/pull/31413#discussion_r568252913 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -591,20 +590,34 @@ case class FileSourceScanExec(

[GitHub] [spark] mridulm edited a comment on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

mridulm edited a comment on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-771271706 I am fine with preserving the behavior - but it looks broken to me. Just to clarify the behavior as I understand - * If there is insufficient resources to run

[GitHub] [spark] mridulm commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

mridulm commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-771271706 I am fine with preserving the behavior - but it looks broken to me. Just to clarify the behavior as I understand - * If there is insufficient resources to run the

[GitHub] [spark] maropu commented on a change in pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

maropu commented on a change in pull request #31427: URL: https://github.com/apache/spark/pull/31427#discussion_r568251053 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -2100,8 +2100,7 @@ class DataSourceV2SQLSuite

[GitHub] [spark] AmplabJenkins commented on pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

AmplabJenkins commented on pull request #31427: URL: https://github.com/apache/spark/pull/31427#issuecomment-771270740 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39335/

[GitHub] [spark] SparkQA commented on pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

SparkQA commented on pull request #31427: URL: https://github.com/apache/spark/pull/31427#issuecomment-771270713 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39335/

[GitHub] [spark] SparkQA commented on pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

SparkQA commented on pull request #31427: URL: https://github.com/apache/spark/pull/31427#issuecomment-771263775 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39335/

[GitHub] [spark] AmplabJenkins commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

AmplabJenkins commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771266963 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39334/

[GitHub] [spark] github-actions[bot] closed pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.0

github-actions[bot] closed pull request #30135: URL: https://github.com/apache/spark/pull/30135 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon closed pull request #31401: [SPARK-34300][PYSPARK][DOCS][MINOR] Fix some typos and syntax issues in docstrings and output of `dev/lint-python`

HyukjinKwon closed pull request #31401: URL: https://github.com/apache/spark/pull/31401 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31401: [SPARK-34300][PYSPARK][DOCS][MINOR] Fix some typos and syntax issues in docstrings and output of `dev/lint-python`

HyukjinKwon commented on pull request #31401: URL: https://github.com/apache/spark/pull/31401#issuecomment-771257865 Merged to master and branch-3.1. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon closed pull request #31408: [SPARK-34306][SQL][PYTHON][R] Use Snake naming rule across the function APIs

HyukjinKwon closed pull request #31408: URL: https://github.com/apache/spark/pull/31408 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31408: [SPARK-34306][SQL][PYTHON][R] Use Snake naming rule across the function APIs

HyukjinKwon commented on pull request #31408: URL: https://github.com/apache/spark/pull/31408#issuecomment-771257354 Thanks all. Merged to master. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #31417: [SPARK-34310][CORE][SQL][2.4] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31417: URL: https://github.com/apache/spark/pull/31417#issuecomment-771256985 Merged to branch-2.4. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon closed pull request #31417: [SPARK-34310][CORE][SQL][2.4] Replaces map and flatten with flatMap

HyukjinKwon closed pull request #31417: URL: https://github.com/apache/spark/pull/31417 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon edited a comment on pull request #31418: [SPARK-34310][CORE][SQL][3.0] Replaces map and flatten with flatMap

HyukjinKwon edited a comment on pull request #31418: URL: https://github.com/apache/spark/pull/31418#issuecomment-771256837 Merged to branch-3.0. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #31418: [SPARK-34310][CORE][SQL][3.0] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31418: URL: https://github.com/apache/spark/pull/31418#issuecomment-771256837 Merged to branch-2.4. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon closed pull request #31418: [SPARK-34310][CORE][SQL][3.0] Replaces map and flatten with flatMap

HyukjinKwon closed pull request #31418: URL: https://github.com/apache/spark/pull/31418 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31416: [SPARK-34310][CORE][SQL] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31416: URL: https://github.com/apache/spark/pull/31416#issuecomment-771256210 Done :-) This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] srowen commented on pull request #31416: [SPARK-34310][CORE][SQL] Replaces map and flatten with flatMap

srowen commented on pull request #31416: URL: https://github.com/apache/spark/pull/31416#issuecomment-771255976 OK, eh I didn't back port the merge to 3.1. The PRs back port to 3.0 / 2.4. It'd be have to backported to 3.1 manually first.

[GitHub] [spark] HyukjinKwon commented on pull request #31417: [SPARK-34310][CORE][SQL][2.4] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31417: URL: https://github.com/apache/spark/pull/31417#issuecomment-771255882 Thanks @srowen. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HyukjinKwon commented on pull request #31416: [SPARK-34310][CORE][SQL] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31416: URL: https://github.com/apache/spark/pull/31416#issuecomment-771255480 Let me just port it back for now, @srowen. It shouldn't affect anything in the codes anyway ... This is

[GitHub] [spark] HyukjinKwon commented on pull request #31416: [SPARK-34310][CORE][SQL] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31416: URL: https://github.com/apache/spark/pull/31416#issuecomment-771254590 Yeah, technically it's not something we should port back since it's neither a bug fix nor an improvement. I just asked him to backport just to reduce a conflict when we

[GitHub] [spark] srowen commented on pull request #31417: [SPARK-34310][CORE][SQL][2.4] Replaces map and flatten with flatMap

srowen commented on pull request #31417: URL: https://github.com/apache/spark/pull/31417#issuecomment-771254512 It won't matter either way I'm sure, so go ahead if anyone would like to. I doubt it's going to matter. This is

[GitHub] [spark] HyukjinKwon commented on pull request #31417: [SPARK-34310][CORE][SQL][2.4] Replaces map and flatten with flatMap

HyukjinKwon commented on pull request #31417: URL: https://github.com/apache/spark/pull/31417#issuecomment-771253503 Nope, it doesn't improve anything. I just suggested him to port back such changes to reduce the conflicts when you backport other changes in the future.

[GitHub] [spark] holdenk commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-771249274 I think I've addressed all of the outstanding issues, if no one has any remaining concerns I'll merge this tomorrow.

[GitHub] [spark] SparkQA commented on pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

SparkQA commented on pull request #31427: URL: https://github.com/apache/spark/pull/31427#issuecomment-771248088 **[Test build #134749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134749/testReport)** for PR 31427 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

AmplabJenkins removed a comment on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771243132 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134743/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

AmplabJenkins removed a comment on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771183445 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

SparkQA removed a comment on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771086398 **[Test build #134743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134743/testReport)** for PR 31422 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

AmplabJenkins removed a comment on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771242166 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39331/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins removed a comment on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771242165 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39332/

[GitHub] [spark] xkrogen commented on pull request #31333: [SPARK-34182][AVRO] Improve error messages when matching Catalyst-to-Avro schemas

xkrogen commented on pull request #31333: URL: https://github.com/apache/spark/pull/31333#issuecomment-771244723 cc also @viirya @bozhang2820 @lindblombr who I see have some fingerprints in relevant files in case they have any feedback

[GitHub] [spark] AmplabJenkins commented on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

AmplabJenkins commented on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771243132 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134743/

[GitHub] [spark] AmplabJenkins commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

AmplabJenkins commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771242166 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39331/

[GitHub] [spark] AmplabJenkins commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771242165 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39332/

[GitHub] [spark] AmplabJenkins commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

AmplabJenkins commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771242164 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134744/

[GitHub] [spark] SparkQA commented on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

SparkQA commented on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771241837 **[Test build #134743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134743/testReport)** for PR 31422 at commit

[GitHub] [spark] SparkQA commented on pull request #31426: [SPARK-34318][SQL] Dataset.colRegex should work with column names and qualifiers which contain newlines

SparkQA commented on pull request #31426: URL: https://github.com/apache/spark/pull/31426#issuecomment-771241003 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39333/

[GitHub] [spark] mridulm commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

mridulm commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-771240887 Late LGTM, thanks for the fix @holdenk ! This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA removed a comment on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

SparkQA removed a comment on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771115304 **[Test build #134744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134744/testReport)** for PR 31423 at commit

[GitHub] [spark] SparkQA commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

SparkQA commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771239522 **[Test build #134744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134744/testReport)** for PR 31423 at commit

[GitHub] [spark] allisonwang-db commented on a change in pull request #31316: [SPARK-33599][SQL][FOLLOWUP] Group exception messages in catalyst/analysis

allisonwang-db commented on a change in pull request #31316: URL: https://github.com/apache/spark/pull/31316#discussion_r568214737 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala ## @@ -732,4 +732,20 @@ private[spark] object

[GitHub] [spark] holdenk opened a new pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

holdenk opened a new pull request #31427: URL: https://github.com/apache/spark/pull/31427 ### What changes were proposed in this pull request? Delegate table name validation to the session catalog ### Why are the changes needed? Queerying of tables with nested

[GitHub] [spark] SparkQA commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

SparkQA commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771228689 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39331/

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771222473 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39332/

[GitHub] [spark] SparkQA commented on pull request #31426: [SPARK-34318][SQL] Dataset.colRegex should work with column names and qualifiers which contain newlines

SparkQA commented on pull request #31426: URL: https://github.com/apache/spark/pull/31426#issuecomment-77128 **[Test build #134747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134747/testReport)** for PR 31426 at commit

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-771216519 > @razajafri, do you mind clarifying PR description? For exmaple, I thought you meant writing out to files or somewhere by: > > > Spark should read it as a long but

[GitHub] [spark] SparkQA commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

SparkQA commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771215978 **[Test build #134748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134748/testReport)** for PR 31423 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31408: [SPARK-34306][SQL][PYTHON][R] Use Snake naming rule across the function APIs

HyukjinKwon commented on a change in pull request #31408: URL: https://github.com/apache/spark/pull/31408#discussion_r568197703 ## File path: R/pkg/R/functions.R ## @@ -636,20 +636,33 @@ setMethod("bin", }) #' @details -#' \code{bitwiseNOT}: Computes bitwise NOT.

[GitHub] [spark] AmplabJenkins commented on pull request #31421: [SPARK-33591][SQL][FOLLOWUP] Add legacy config for recognizing null partition spec values

AmplabJenkins commented on pull request #31421: URL: https://github.com/apache/spark/pull/31421#issuecomment-771214976 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134741/

[GitHub] [spark] AmplabJenkins commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

AmplabJenkins commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-771214977 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134742/

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771212397 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39332/

[GitHub] [spark] SparkQA commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

SparkQA commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771212229 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39331/

[GitHub] [spark] SparkQA removed a comment on pull request #31421: [SPARK-33591][SQL][FOLLOWUP] Add legacy config for recognizing null partition spec values

SparkQA removed a comment on pull request #31421: URL: https://github.com/apache/spark/pull/31421#issuecomment-771045274 **[Test build #134741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134741/testReport)** for PR 31421 at commit

[GitHub] [spark] SparkQA commented on pull request #31421: [SPARK-33591][SQL][FOLLOWUP] Add legacy config for recognizing null partition spec values

SparkQA commented on pull request #31421: URL: https://github.com/apache/spark/pull/31421#issuecomment-771208942 **[Test build #134741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134741/testReport)** for PR 31421 at commit

[GitHub] [spark] sarutak opened a new pull request #31426: [SPARK-34318][SQL] Dataset.colRegex should work with column names and qualifiers which contain newlines

sarutak opened a new pull request #31426: URL: https://github.com/apache/spark/pull/31426 ### What changes were proposed in this pull request? This PR fixes an issue that `Dataset.colRegex` doesn't work with column names or qualifiers which contain newlines. In the current

[GitHub] [spark] SparkQA removed a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

SparkQA removed a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-771037753 **[Test build #134742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134742/testReport)** for PR 31284 at commit

[GitHub] [spark] SparkQA commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

SparkQA commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-771194064 **[Test build #134742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134742/testReport)** for PR 31284 at commit

[GitHub] [spark] SparkQA commented on pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

SparkQA commented on pull request #31424: URL: https://github.com/apache/spark/pull/31424#issuecomment-771189398 **[Test build #134745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134745/testReport)** for PR 31424 at commit

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-771186832 **[Test build #134746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134746/testReport)** for PR 31413 at commit

[GitHub] [spark] imback82 commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

imback82 commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r568167843 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/udaf.scala ## @@ -325,13 +325,16 @@ case class ScalaUDAF( children:

[GitHub] [spark] imback82 commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

imback82 commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r568167394 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ## @@ -159,11 +159,19 @@ class CacheManager extends Logging

[GitHub] [spark] imback82 commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

imback82 commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r568167394 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ## @@ -159,11 +159,19 @@ class CacheManager extends Logging

[GitHub] [spark] c21 commented on a change in pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

c21 commented on a change in pull request #31413: URL: https://github.com/apache/spark/pull/31413#discussion_r568161605 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -591,20 +590,34 @@ case class FileSourceScanExec(

[GitHub] [spark] AmplabJenkins commented on pull request #31425: [WIP][SPARK-34315] Stops debconf asking for default kerberos realm

AmplabJenkins commented on pull request #31425: URL: https://github.com/apache/spark/pull/31425#issuecomment-771183660 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

AmplabJenkins commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771183445 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39330/

[GitHub] [spark] timhughes opened a new pull request #31425: [WIP][SPARK-34315] Stops debconf asking for default kerberos realm

timhughes opened a new pull request #31425: URL: https://github.com/apache/spark/pull/31425 https://issues.apache.org/jira/browse/SPARK-34315 ### What changes were proposed in this pull request? This makes a change to prevent debconf from configuring Kerberos

[GitHub] [spark] ekoifman commented on pull request #30829: [SPARK-33832][SQL] Add an option in AQE to mitigate skew even if it c…

ekoifman commented on pull request #30829: URL: https://github.com/apache/spark/pull/30829#issuecomment-771170612 The main loop in `AdaptiveSparkPlanExec.getFinalPhysicalPlan()` has this check ``` if (newCost < origCost || (newCost == origCost &&

[GitHub] [spark] imback82 commented on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

imback82 commented on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771162172 cc @cloud-fan @MaxGekk This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] imback82 opened a new pull request #31424: [SPARK-34317][SQL] Introduce relationTypeMismatchHint to UnresolvedTable for a better error message

imback82 opened a new pull request #31424: URL: https://github.com/apache/spark/pull/31424 ### What changes were proposed in this pull request? This PR proposes to add `relationTypeMismatchHint` to `UnresolvedTable` so that if a relation is resolved to a view when a table is

[GitHub] [spark] SparkQA commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

SparkQA commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771156648 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39330/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

AmplabJenkins removed a comment on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771149108 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39329/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly

AmplabJenkins removed a comment on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-771149110 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134739/

[GitHub] [spark] AmplabJenkins commented on pull request #31422: [SPARK-34313][SQL] Migrate ALTER TABLE SET/UNSET TBLPROPERTIES commands to use UnresolvedTable to resolve the identifier

AmplabJenkins commented on pull request #31422: URL: https://github.com/apache/spark/pull/31422#issuecomment-771149108 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39329/

[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly

AmplabJenkins commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-771149110 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134739/

[GitHub] [spark] SparkQA commented on pull request #31423: [SPARK-34314][SQL] Rebuild the file index after partition schema inferring

SparkQA commented on pull request #31423: URL: https://github.com/apache/spark/pull/31423#issuecomment-771146144 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39330/

<    1   2   3   4   5   6   7   8   >