[PR] [SPARK-46547] Fix deadlock between maintenance thread and streaming aggregation operator [spark]

anishshri-db opened a new pull request, #44542: URL: https://github.com/apache/spark/pull/44542 ### What changes were proposed in this pull request? Fix deadlock between maintenance thread and streaming aggregation operator ### Why are the changes needed? This change fixes a race

Re: [PR] [SPARK-46490][SQL] Require error classes in `SparkThrowable` sub-classes [spark]

MaxGekk commented on PR #44464: URL: https://github.com/apache/spark/pull/44464#issuecomment-1872491581 Merging to master. Thank you, @heyihong and @dongjoon-hyun for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [SPARK-46490][SQL] Require error classes in `SparkThrowable` sub-classes [spark]

MaxGekk closed pull request #44464: [SPARK-46490][SQL] Require error classes in `SparkThrowable` sub-classes URL: https://github.com/apache/spark/pull/44464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [SPARK-46547][SS] Fix deadlock between maintenance thread and streaming aggregation operator [spark]

anishshri-db commented on PR #44542: URL: https://github.com/apache/spark/pull/44542#issuecomment-1872492772 cc - @HeartSaVioR - PTAL whenever you get a chance, thx ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [SPARK-46539][SQL] SELECT * EXCEPT(all fields from a struct) results in an assertion failure [spark]

MaxGekk commented on PR #44527: URL: https://github.com/apache/spark/pull/44527#issuecomment-1872498889 @stefankandic Just to double check, up to which version should the changes be ported? The ticket SPARK-46539 points out 3.0.0, is it correct? -- This is an automated message from the A

Re: [PR] [SPARK-46541][SQL][CONNECT] Fix the ambiguous column reference in self join [spark]

zhengruifeng commented on PR #44532: URL: https://github.com/apache/spark/pull/44532#issuecomment-1872499783 cc @cloud-fan I think it is ready for the initial review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [SPARK-46541][SQL][CONNECT] Fix the ambiguous column reference in self join [spark]

zhengruifeng commented on code in PR #44532: URL: https://github.com/apache/spark/pull/44532#discussion_r1438547075 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -520,9 +524,33 @@ trait ColumnResolutionHelper extends Lo

Re: [PR] [SPARK-46541][SQL][CONNECT] Fix the ambiguous column reference in self join [spark]

zhengruifeng commented on code in PR #44532: URL: https://github.com/apache/spark/pull/44532#discussion_r1438547075 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -520,9 +524,33 @@ trait ColumnResolutionHelper extends Lo

Re: [PR] debug appveyor [spark]

panbingkun closed pull request #44541: debug appveyor URL: https://github.com/apache/spark/pull/44541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

Re: [PR] [SPARK-46536][SQL] Support GROUP BY calendar_interval_type [spark]

beliefer commented on code in PR #44538: URL: https://github.com/apache/spark/pull/44538#discussion_r1438554749 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/CalendarInterval.java: ## @@ -44,7 +44,7 @@ * @since 3.0.0 */ @Unstable -public final class CalendarI

[PR] [SPARK-46504][PS][TESTS][FOLLOWUPS] Moving move slow tests out of `IndexesTests` [spark]

zhengruifeng opened a new pull request, #44543: URL: https://github.com/apache/spark/pull/44543 ### What changes were proposed in this pull request? Moving move slow tests out of `IndexesTests` ### Why are the changes needed? for testing parallelism ### Does this P

[PR] test slf4j-api 2.0.10 [spark]

panbingkun opened a new pull request, #44544: URL: https://github.com/apache/spark/pull/44544 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

Re: [PR] [SPARK-46331][SQL] Removing CodegenFallback from subset of DateTime expressions and version() expression [spark]

beliefer commented on code in PR #44261: URL: https://github.com/apache/spark/pull/44261#discussion_r1438580819 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala: ## @@ -78,31 +81,14 @@ class DateExpressionsSuite extends SparkFun

Re: [PR] [SPARK-46544][SQL] Support v2 DESCRIBE TABLE EXTENDED with table stats [spark]

Zouxxyy commented on PR #44535: URL: https://github.com/apache/spark/pull/44535#issuecomment-1872520469 @MaxGekk Thanks for your reminder, fix it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] [SPARK-46547][SS] Fix deadlock between maintenance thread and streaming aggregation operator [spark]

anishshri-db commented on code in PR #44542: URL: https://github.com/apache/spark/pull/44542#discussion_r1438619998 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala: ## @@ -434,22 +434,26 @@ case class StateStoreRestoreExec( numC

Re: [PR] [SPIP-IN-PROGRESS][DO-NOT-MERGE][SS] Add base support for new arbitrary state management operator, single valueState type, multiple state variables and underlying support for column families

anishshri-db commented on code in PR #43961: URL: https://github.com/apache/spark/pull/43961#discussion_r1438622114 ## sql/api/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/TransformWithStateTimeoutModes.scala: ## @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Soft

[PR] [SPARK-46548][PYTHON][DOCS] Refine docstring of `get/array_zip/sort_array` [spark]

LuciferYang opened a new pull request, #44545: URL: https://github.com/apache/spark/pull/44545 ### What changes were proposed in this pull request? This pr refine docstring of `get/array_zip/sort_array` and add some new examples. ### Why are the changes needed? To improve PySpar

Re: [PR] [SPARK-46548][PYTHON][DOCS] Refine docstring of `get/array_zip/sort_array` [spark]

LuciferYang commented on PR #44545: URL: https://github.com/apache/spark/pull/44545#issuecomment-1872574701 Check ansi is true first -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] [SPARK-46536][SQL] Support GROUP BY calendar_interval_type [spark]

srielau commented on PR #44538: URL: https://github.com/apache/spark/pull/44538#issuecomment-1872579314 We (i.e. @MaxGekk) have added standard year-month and day-time intervals which are much better than calendar interval. It is of questionable value to improve this old type. @cloud-fa

Re: [PR] WIP: refactor Window operator [spark]

github-actions[bot] commented on PR #42963: URL: https://github.com/apache/spark/pull/42963#issuecomment-1872634179 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [Don't merge and review] investigate root cause sbt [spark]

github-actions[bot] commented on PR #42961: URL: https://github.com/apache/spark/pull/42961#issuecomment-1872634186 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [MiNOR][DOCS] Fix a typo in HashAggregateExec.scala [spark]

github-actions[bot] closed pull request #42916: [MiNOR][DOCS] Fix a typo in HashAggregateExec.scala URL: https://github.com/apache/spark/pull/42916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[PR] [WIP][SPARK-46549][INFRA] Cache the Python dependencies for SQL tests [spark]

zhengruifeng opened a new pull request, #44546: URL: https://github.com/apache/spark/pull/44546 ### What changes were proposed in this pull request? Enable the caching provided by [`setup-python`](https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#caching-packages)

Re: [PR] [WIP][SPARK-46549][INFRA] Cache the Python dependencies for SQL tests [spark]

zhengruifeng commented on code in PR #44546: URL: https://github.com/apache/spark/pull/44546#discussion_r1438768278 ## dev/py-tests/requirements-sql.txt: ## @@ -0,0 +1,11 @@ +# PySpark dependencies for SQL tests + +numpy==1.26.2 Review Comment: > The requirements file format

Re: [PR] [SPARK-46548][PYTHON][DOCS] Refine docstring of `get/array_zip/sort_array` [spark]

LuciferYang closed pull request #44545: [SPARK-46548][PYTHON][DOCS] Refine docstring of `get/array_zip/sort_array` URL: https://github.com/apache/spark/pull/44545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] [SPARK-46548][PYTHON][DOCS] Refine docstring of `get/array_zip/sort_array` [spark]

LuciferYang commented on PR #44545: URL: https://github.com/apache/spark/pull/44545#issuecomment-1872800926 Merged into master. Thanks @zhengruifeng -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] test slf4j-api 2.0.10 [spark]

LuciferYang commented on PR #44544: URL: https://github.com/apache/spark/pull/44544#issuecomment-1872803177 Cloud you update the this pr? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci