Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

HeartSaVioR closed pull request #43788: [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation URL: https://github.com/apache/spark/pull/43788 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

HeartSaVioR commented on PR #43788: URL: https://github.com/apache/spark/pull/43788#issuecomment-1812633610 @chaoqin-li1123 Could you please rebase your change with latest master branch? merge script is confusing that I'm the main author due to my commits listed here. -- This is an

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

HeartSaVioR commented on PR #43788: URL: https://github.com/apache/spark/pull/43788#issuecomment-1812628737 Thanks! Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

chaoqin-li1123 commented on code in PR #43788: URL: https://github.com/apache/spark/pull/43788#discussion_r1393180207 ## sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/v2/state/StateDataSourceReadSuite.scala: ## @@ -494,6 +497,48 @@ abstract class

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

chaoqin-li1123 commented on code in PR #43788: URL: https://github.com/apache/spark/pull/43788#discussion_r1393179190 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/StatePartitionReader.scala: ## @@ -57,12 +58,31 @@ class StatePartitionReader(

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

chaoqin-li1123 commented on code in PR #43788: URL: https://github.com/apache/spark/pull/43788#discussion_r1393178895 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/metadata/StateMetadataSource.scala: ## @@ -195,7 +195,7 @@ class

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

chaoqin-li1123 commented on code in PR #43788: URL: https://github.com/apache/spark/pull/43788#discussion_r1393176718 ## sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/v2/state/StateDataSourceReadSuite.scala: ## @@ -494,6 +497,48 @@ abstract class

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

HeartSaVioR commented on code in PR #43788: URL: https://github.com/apache/spark/pull/43788#discussion_r1392113944 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/StatePartitionReader.scala: ## @@ -57,12 +58,31 @@ class StatePartitionReader(

Re: [PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

chaoqin-li1123 commented on PR #43788: URL: https://github.com/apache/spark/pull/43788#issuecomment-1809278646 @HeartSaVioR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation [spark]

chaoqin-li1123 opened a new pull request, #43788: URL: https://github.com/apache/spark/pull/43788 ### What changes were proposed in this pull request? Currently reading state for session window aggregation operator is not supported because the numColPrefixKey is unknown. We can