rluvaton commented on code in PR #24017:
URL: https://github.com/apache/datafusion/pull/24017#discussion_r3913140909
##########
datafusion/physical-plan/src/aggregates/hash_stream.rs:
##########
@@ -249,47 +210,15 @@ pub(crate) struct FinalHashAggregateStream {
/// See comments for the same variable in [`PartialHashAggregateStream`].
group_values_soft_limit: Option<usize>,
- /// Tracks the high-level stream lifecycle. The hash table owns the
lower-level
+ /// The hash table owns the lower-level
/// state for emitting output batches.
- state: Option<FinalHashAggregateState>,
-}
-
-/// States for final hash aggregation processing.
-// The typestate pattern is used in case the inner logic becomes more complex
in
-// the future.
-enum FinalHashAggregateState {
Review Comment:
the state machine is unneded since it is almost linear with async generators
for the final stream
you read from input, if you dont have memory, spill and continue reading
once finished, if have spills, spill the in pending data in hash table and
switch to ordered final aggregate stream
if no spills, just output batches from hash table.
<details>
<summary>Flow</summary>
<img width="3289" height="6420" alt="image"
src="https://github.com/user-attachments/assets/d5b914f6-8f53-4576-8b7d-cb8ff6351e9c";
/>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
