[
https://issues.apache.org/jira/browse/DRILL-5457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055037#comment-16055037
]
ASF GitHub Bot commented on DRILL-5457:
---------------------------------------
Github user Ben-Zvi commented on a diff in the pull request:
https://github.com/apache/drill/pull/822#discussion_r122863244
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
---
@@ -149,14 +149,24 @@ public IterOutcome innerNext() {
if ( aggOut == HashAggregator.AggIterOutcome.AGG_OK ) { return
IterOutcome.OK; }
// if RESTART - continue below with doWork() - read some spilled
partition, just like reading incoming
incoming = aggregator.getNewIncoming(); // Restart - incoming was
just changed
- if ( wasKilled ) { // if kill() was called before, then finish up
- aggregator.cleanup();
- incoming.kill(false);
- return IterOutcome.NONE;
- }
}
- AggOutcome out = aggregator.doWork();
+ if ( wasKilled ) { // if kill() was called before, then finish up
+ aggregator.cleanup();
+ incoming.kill(false);
+ return IterOutcome.NONE;
+ }
+
+ // Read and aggregate records
+ // ( may need to run again if the spilled partition that was read
+ // generated new partitions that were all spilled )
+ AggOutcome out = AggOutcome.CALL_WORK_AGAIN;
+ while ( out == AggOutcome.CALL_WORK_AGAIN) {
+ //
+ // Read incoming batches and process their records
+ //
+ out = aggregator.doWork();
+ }
--- End diff --
Done ( do while ...)
> Support Spill to Disk for the Hash Aggregate Operator
> -----------------------------------------------------
>
> Key: DRILL-5457
> URL: https://issues.apache.org/jira/browse/DRILL-5457
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.10.0
> Reporter: Boaz Ben-Zvi
> Assignee: Boaz Ben-Zvi
> Fix For: 1.11.0
>
>
> Support gradual spilling memory to disk as the available memory gets too
> small to allow in memory work for the Hash Aggregate Operator.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
