[GitHub] [spark] SparkQA removed a comment on pull request #31316: [SPARK-33599][SQL][FOLLOWUP] Group exception messages in catalyst/analysis

SparkQA removed a comment on pull request #31316: URL: https://github.com/apache/spark/pull/31316#issuecomment-776512053 **[Test build #135095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135095/testReport)** for PR 31316 at commit

[GitHub] [spark] SparkQA commented on pull request #31316: [SPARK-33599][SQL][FOLLOWUP] Group exception messages in catalyst/analysis

SparkQA commented on pull request #31316: URL: https://github.com/apache/spark/pull/31316#issuecomment-776514201 **[Test build #135095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135095/testReport)** for PR 31316 at commit

[GitHub] [spark] SparkQA commented on pull request #31542: [SPARK-34414][SQL] OptimizeMetadataOnlyQuery should only apply for deterministic filters

SparkQA commented on pull request #31542: URL: https://github.com/apache/spark/pull/31542#issuecomment-776512107 **[Test build #135093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135093/testReport)** for PR 31542 at commit

[GitHub] [spark] SparkQA commented on pull request #31316: [SPARK-33599][SQL][FOLLOWUP] Group exception messages in catalyst/analysis

SparkQA commented on pull request #31316: URL: https://github.com/apache/spark/pull/31316#issuecomment-776512053 **[Test build #135095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135095/testReport)** for PR 31316 at commit

[GitHub] [spark] SparkQA commented on pull request #31541: Revert "[SPARK-34209][SQL] Delegate table name validation to the session catalog"

SparkQA commented on pull request #31541: URL: https://github.com/apache/spark/pull/31541#issuecomment-776511932 **[Test build #135094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135094/testReport)** for PR 31541 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

HyukjinKwon commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r573502020 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -1141,6 +1141,36 @@ trait AlterTableTests extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

AmplabJenkins removed a comment on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776503691 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135091/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776503690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135088/

[GitHub] [spark] gaborgsomogyi commented on pull request #31384: [SPARK-31816][SQL][DOCS] Added high level description about JDBC connection providers for users/developers

gaborgsomogyi commented on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-776504404 @HyukjinKwon @maropu @HeartSaVioR Thank you for taking care! With this last PR I consider the feature done from my side.

[GitHub] [spark] AmplabJenkins commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

AmplabJenkins commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776503691 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135091/

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776503690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135088/

[GitHub] [spark] HyukjinKwon commented on pull request #31542: [MINOR][SQL] OptimizeMetadataOnlyQuery should only apply for deterministic filters

HyukjinKwon commented on pull request #31542: URL: https://github.com/apache/spark/pull/31542#issuecomment-776503505 @yeshengm do you mind filing a JIRA please? This is an automated message from the Apache Git Service. To

[GitHub] [spark] yeshengm opened a new pull request #31542: [MINOR][SQL] OptimizeMetadataOnlyQuery should only apply for deterministic filters

yeshengm opened a new pull request #31542: URL: https://github.com/apache/spark/pull/31542 ### What changes were proposed in this pull request? ### Why are the changes needed? Similar to FileSourcePartitionPruning, OptimizeMetadataOnlyQuery should only

[GitHub] [spark] SparkQA removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776412166 **[Test build #135088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135088/testReport)** for PR 31337 at commit

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776497711 **[Test build #135088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135088/testReport)** for PR 31337 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

SparkQA removed a comment on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776430570 **[Test build #135091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135091/testReport)** for PR 31442 at commit

[GitHub] [spark] SparkQA commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

SparkQA commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776491691 **[Test build #135091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135091/testReport)** for PR 31442 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #31541: Revert "[SPARK-34209][SQL] Delegate table name validation to the session catalog"

HyukjinKwon commented on pull request #31541: URL: https://github.com/apache/spark/pull/31541#issuecomment-776486423 @holdenk, @cloud-fan, @imback82, @dongjoon-hyun, @maropu, I opened a PR to discuss the revert of https://github.com/apache/spark/pull/31427. Could you guys take a look

[GitHub] [spark] HyukjinKwon opened a new pull request #31541: Revert "[SPARK-34209][SQL] Delegate table name validation to the session catalog"

HyukjinKwon opened a new pull request #31541: URL: https://github.com/apache/spark/pull/31541 ### What changes were proposed in this pull request? This reverts commit cf7a13c363ef5d56556c9d70e7811bf6a40de55f. ### Why are the changes needed? To fix the regression in the

[GitHub] [spark] MaxGekk commented on pull request #31499: [SPARK-31891][SQL] Support `MSCK REPAIR TABLE .. [ADD|DROP|SYNC] PARTITIONS`

MaxGekk commented on pull request #31499: URL: https://github.com/apache/spark/pull/31499#issuecomment-776473007 @dongjoon-hyun @viirya Are you ok with the changes? This is an automated message from the Apache Git Service.

[GitHub] [spark] zzcclp commented on a change in pull request #31404: [SPARK-34283][SQL] Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct.union.distinct

zzcclp commented on a change in pull request #31404: URL: https://github.com/apache/spark/pull/31404#discussion_r573482876 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1007,6 +1007,8 @@ object CombineUnions extends

[GitHub] [spark] zzcclp commented on pull request #31404: [SPARK-34283][SQL] Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct.union.distinct'

zzcclp commented on pull request #31404: URL: https://github.com/apache/spark/pull/31404#issuecomment-776482363 > Could you update the PR description along with the current approach? Done This is an automated message

[GitHub] [spark] beliefer commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

beliefer commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776481714 @cloud-fan Thanks for your work! @maropu @viirya Thanks for your review! This is an automated message from the

[GitHub] [spark] cloud-fan closed pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

cloud-fan closed pull request #31529: URL: https://github.com/apache/spark/pull/31529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

cloud-fan commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-776476030 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

cloud-fan closed pull request #31337: URL: https://github.com/apache/spark/pull/31337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

cloud-fan commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776477090 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

AmplabJenkins removed a comment on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776476510 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39674/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

AmplabJenkins removed a comment on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776476511 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39673/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776476508 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39671/

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776476508 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39671/

[GitHub] [spark] AmplabJenkins commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

AmplabJenkins commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776476510 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39674/

[GitHub] [spark] AmplabJenkins commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

AmplabJenkins commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776476511 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39673/

[GitHub] [spark] LuciferYang commented on pull request #31528: [SPARK-34403][SQL]Remove dependency to commons-httpclient

LuciferYang commented on pull request #31528: URL: https://github.com/apache/spark/pull/31528#issuecomment-776475281 > hive-exec need this dependency. Will this class actually be loaded by the JVM? Need to wait hive upgrades this dependency first?

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31538: [SPARK-34412][SQL] RemoveNoopOperators should only remove trivial projects

dongjoon-hyun commented on a change in pull request #31538: URL: https://github.com/apache/spark/pull/31538#discussion_r573472762 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RemoveNoopOperatorsSuite.scala ## @@ -44,6 +45,22 @@ class

[GitHub] [spark] SparkQA commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

SparkQA commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776468501 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39674/

[GitHub] [spark] MaxGekk commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

MaxGekk commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-776460420 @cloud-fan @gengliangwang @HyukjinKwon This PR is a companion to https://github.com/apache/spark/pull/31489, and should solve the same issues. Any concerns about it?

[GitHub] [spark] SparkQA commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

SparkQA commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776460281 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39673/

[GitHub] [spark] MaxGekk commented on pull request #31499: [SPARK-31891][SQL] Support `MSCK REPAIR TABLE .. [ADD|DROP|SYNC] PARTITIONS`

MaxGekk commented on pull request #31499: URL: https://github.com/apache/spark/pull/31499#issuecomment-776459632 @cloud-fan @HyukjinKwon Any objections to the changes? This is an automated message from the Apache Git

[GitHub] [spark] maropu commented on a change in pull request #31404: [SPARK-34283][SQL] Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct.union.distinct

maropu commented on a change in pull request #31404: URL: https://github.com/apache/spark/pull/31404#discussion_r573462571 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1007,6 +1007,8 @@ object CombineUnions extends

[GitHub] [spark] HyukjinKwon commented on pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

HyukjinKwon commented on pull request #31427: URL: https://github.com/apache/spark/pull/31427#issuecomment-776457314 Hey, I am still lost why we needed to change this. This is a regression in the error message, but I am not clear what's the gain here @dongjoon-hyun too.

[GitHub] [spark] cloud-fan commented on a change in pull request #31404: [SPARK-34283][SQL] Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct.union.disti

cloud-fan commented on a change in pull request #31404: URL: https://github.com/apache/spark/pull/31404#discussion_r573461209 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1007,6 +1007,8 @@ object CombineUnions

[GitHub] [spark] SparkQA commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

SparkQA commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776453331 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39674/

[GitHub] [spark] SparkQA commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

SparkQA commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776452940 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39673/

[GitHub] [spark] zhengruifeng commented on pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in UnivariateFeatureSelector document

zhengruifeng commented on pull request #31531: URL: https://github.com/apache/spark/pull/31531#issuecomment-776452308 Late LGTM This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29818: [SPARK-32953][PYTHON] Add Arrow self_destruct support to toPandas

AmplabJenkins removed a comment on pull request #29818: URL: https://github.com/apache/spark/pull/29818#issuecomment-776451202 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135086/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

AmplabJenkins removed a comment on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776451204 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39672/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776451205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39670/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

AmplabJenkins removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776451201 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29818: [SPARK-32953][PYTHON] Add Arrow self_destruct support to toPandas

AmplabJenkins commented on pull request #29818: URL: https://github.com/apache/spark/pull/29818#issuecomment-776451202 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135086/

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776451205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39670/

[GitHub] [spark] AmplabJenkins commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

AmplabJenkins commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776451204 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39672/

[GitHub] [spark] AmplabJenkins commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

AmplabJenkins commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776451201 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

cloud-fan commented on pull request #31427: URL: https://github.com/apache/spark/pull/31427#issuecomment-776449191 @holdenk Can we make sure PRs are merged with at least one approval from committers? And also please enrich the PR description a bit more: I don't see where the delegation

[GitHub] [spark] SparkQA commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

SparkQA commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776447930 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39672/

[GitHub] [spark] SparkQA commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

SparkQA commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776446396 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39672/

[GitHub] [spark] cloud-fan commented on a change in pull request #31427: [SPARK-34209][SQL] Delegate table name validation to the session catalog

cloud-fan commented on a change in pull request #31427: URL: https://github.com/apache/spark/pull/31427#discussion_r573453436 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -2216,27 +2213,46 @@ class DataSourceV2SQLSuite

[GitHub] [spark] SparkQA removed a comment on pull request #29818: [SPARK-32953][PYTHON] Add Arrow self_destruct support to toPandas

SparkQA removed a comment on pull request #29818: URL: https://github.com/apache/spark/pull/29818#issuecomment-776347207 **[Test build #135086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135086/testReport)** for PR 29818 at commit

[GitHub] [spark] SparkQA commented on pull request #29818: [SPARK-32953][PYTHON] Add Arrow self_destruct support to toPandas

SparkQA commented on pull request #29818: URL: https://github.com/apache/spark/pull/29818#issuecomment-776444897 **[Test build #135086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135086/testReport)** for PR 29818 at commit

[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776444351 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39669/

[GitHub] [spark] SparkQA removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

SparkQA removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776398704 **[Test build #135087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135087/testReport)** for PR 31450 at commit

[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776443862 **[Test build #135087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135087/testReport)** for PR 31450 at commit

[GitHub] [spark] mridulm edited a comment on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

mridulm edited a comment on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-776442655 This does not necessarily solve the issue that @zsxwing detailed - the issue here is `registerAccumulator` should not be called in `readObject` before subclasses have

[GitHub] [spark] mridulm edited a comment on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

mridulm edited a comment on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-776442655 This does not necessarily solve the issue that @zsxwing detailed - the issue here is `registerAccumulator` should not be called in `readObject` before subclasses have

[GitHub] [spark] mridulm commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

mridulm commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-776442655 This does not necessarily solve the issue that @zsxwing detailed - the issue here is `registerAccumulator` should not be called in `readObject` before subclasses have completed

[GitHub] [spark] SparkQA commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

SparkQA commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776440924 **[Test build #135092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135092/testReport)** for PR 31468 at commit

[GitHub] [spark] attilapiros edited a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

attilapiros edited a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776439581 > Wonderful,... Especially after it's been spiced up with my painting skills :) This is an

[GitHub] [spark] viirya closed pull request #31462: [SPARK-34347][SQL] CatalogImpl.uncacheTable should invalidate in cascade for temp views

viirya closed pull request #31462: URL: https://github.com/apache/spark/pull/31462 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] attilapiros commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

attilapiros commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776439581 > Wonderful, thanks for picking this up :) Especially after it's been spiced up with my painting skills

[GitHub] [spark] viirya commented on pull request #31462: [SPARK-34347][SQL] CatalogImpl.uncacheTable should invalidate in cascade for temp views

viirya commented on pull request #31462: URL: https://github.com/apache/spark/pull/31462#issuecomment-776439223 Thanks! Merging to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776434393 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39670/

[GitHub] [spark] SparkQA commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

SparkQA commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776434334 **[Test build #135090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135090/testReport)** for PR 31468 at commit

[GitHub] [spark] holdenk commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

holdenk commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776432824 Wonderful, thanks for picking this up :) This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] mridulm edited a comment on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

mridulm edited a comment on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-776431823 The changes look good to me, thanks @Ngone51 ! Can you merge it when your comments are addressed @tgravescs ? Thx

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

AmplabJenkins removed a comment on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-776151186 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135075/

[GitHub] [spark] mridulm commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

mridulm commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-776431823 The changes look good to me, thanks @Ngone51 ! This is an automated message from the Apache Git Service. To

[GitHub] [spark] zhengruifeng commented on a change in pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on a change in pull request #31468: URL: https://github.com/apache/spark/pull/31468#discussion_r573438154 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ## @@ -200,28 +210,32 @@ case class TakeOrderedAndProjectExec(

[GitHub] [spark] mridulm commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

mridulm commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-776431450 I agree, I think it is fine. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

SparkQA commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776430570 **[Test build #135091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135091/testReport)** for PR 31442 at commit

[GitHub] [spark] maropu commented on a change in pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

maropu commented on a change in pull request #31468: URL: https://github.com/apache/spark/pull/31468#discussion_r573435246 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ## @@ -200,28 +210,32 @@ case class TakeOrderedAndProjectExec(

[GitHub] [spark] sarutak commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

sarutak commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776427904 @maropu Thanks. I've updated. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] zhengruifeng commented on a change in pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on a change in pull request #31468: URL: https://github.com/apache/spark/pull/31468#discussion_r573433676 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ## @@ -52,16 +52,21 @@ case class CollectLimitExec(limit: Int,

[GitHub] [spark] maropu commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

maropu commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-776425851 Could you add tests in the `core` package? This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] zhengruifeng commented on a change in pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on a change in pull request #31468: URL: https://github.com/apache/spark/pull/31468#discussion_r573432645 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ## @@ -200,7 +205,7 @@ case class TakeOrderedAndProjectExec(

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776425676 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39670/

[GitHub] [spark] maropu commented on a change in pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

maropu commented on a change in pull request #31491: URL: https://github.com/apache/spark/pull/31491#discussion_r573431650 ## File path: docs/sql-migration-guide.md ## @@ -24,6 +24,8 @@ license: | ## Upgrading from Spark SQL 3.1 to 3.2 + - Since Spark 3.2, all the

[GitHub] [spark] maropu commented on a change in pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

maropu commented on a change in pull request #31442: URL: https://github.com/apache/spark/pull/31442#discussion_r573431597 ## File path: docs/sql-migration-guide.md ## @@ -24,6 +24,8 @@ license: | ## Upgrading from Spark SQL 3.1 to 3.2 + - In Spark 3.2, PostgreSQL JDBC

[GitHub] [spark] attilapiros commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation

attilapiros commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776422517 cc. @holdenk This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-776419676 **[Test build #135089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135089/testReport)** for PR 31337 at commit

[GitHub] [spark] maropu closed pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

maropu closed pull request #31500: URL: https://github.com/apache/spark/pull/31500 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] maropu commented on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

maropu commented on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-776419198 It seems this PR already has been merged, so I'll close this. This is an automated message from the Apache Git

[GitHub] [spark] maropu commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

maropu commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-776418535 Could you resolve the conflict? Looks fine otherwise. This is an automated message from the Apache Git Service.

[GitHub] [spark] wangyum commented on pull request #31528: [SPARK-34403][SQL]Remove dependency to commons-httpclient

wangyum commented on pull request #31528: URL: https://github.com/apache/spark/pull/31528#issuecomment-776414106 [`hive-exec`](https://github.com/apache/hive/blob/rel/release-2.3.8/ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java#L34) need this dependency.

[GitHub] [spark] maropu commented on pull request #31384: [SPARK-31816][SQL][DOCS] Added high level description about JDBC connection providers for users/developers

maropu commented on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-776413210 Thanks! Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] maropu closed pull request #31384: [SPARK-31816][SQL][DOCS] Added high level description about JDBC connection providers for users/developers

maropu closed pull request #31384: URL: https://github.com/apache/spark/pull/31384 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-776413082 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39669/

[GitHub] [spark] cloud-fan closed pull request #31073: [SPARK-33995][SQL] Expose make_interval as a Scala function

cloud-fan closed pull request #31073: URL: https://github.com/apache/spark/pull/31073 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #31073: [SPARK-33995][SQL] Expose make_interval as a Scala function

cloud-fan commented on pull request #31073: URL: https://github.com/apache/spark/pull/31073#issuecomment-776412824 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] maropu commented on pull request #31384: [SPARK-31816][SQL][DOCS] Added high level description about JDBC connection providers for users/developers

maropu commented on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-776412764 Sorry for my late reply... yea. thanks for the work, @gaborgsomogyi . Looks good now. I'll merge this. This is

[GitHub] [spark] AngersZhuuuu commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AngersZh commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-776412512 Gentle ping @maropu @cloud-fan This is an automated message from the Apache Git Service. To respond to

  1   2   3   4   >