[
https://issues.apache.org/jira/browse/DRILL-5457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033833#comment-16033833
]
ASF GitHub Bot commented on DRILL-5457:
---------------------------------------
Github user Ben-Zvi commented on a diff in the pull request:
https://github.com/apache/drill/pull/822#discussion_r119745935
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
---
@@ -285,8 +648,18 @@ public AggOutcome doWork() {
// In the future HashAggregate may also need to perform some actions
conditionally
// in the outer try block.
+ assert ! handlingSpills || currentIndex < Integer.MAX_VALUE;
+
outside:
while (true) {
+
+ // This would be called only once - after actual data arrives on
incoming
+ if ( schema == null && incoming.getRecordCount() > 0 ) {
--- End diff --
There is no code for OK_FIRST_NON_EMPTY; and the local field "schema" is
used here as a flag to note "setup not yet performed" ( not always matched with
OK_NEW_SCHEMA; sometimes the second batch with an OK is the first non empty
batch).
And next() is a FINAL method (in AbstractRecordBatch), which in turn
invokes other next() methods of other classes extending RecordBatch (like the
new SpilledRecordBatch). Should we put there the code to perform delayed setup
for the HashAgg ?
Even if the next() is modified to return a new flag like
OK_FIRST_NON_EMPTY -- these flags are checked in the code below, starting from
the second batch and on. Not sure where the code reading the first incoming
batch is ....
> Support Spill to Disk for the Hash Aggregate Operator
> -----------------------------------------------------
>
> Key: DRILL-5457
> URL: https://issues.apache.org/jira/browse/DRILL-5457
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.10.0
> Reporter: Boaz Ben-Zvi
> Assignee: Boaz Ben-Zvi
> Fix For: 1.11.0
>
>
> Support gradual spilling memory to disk as the available memory gets too
> small to allow in memory work for the Hash Aggregate Operator.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
