[
https://issues.apache.org/jira/browse/HIVE-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13750638#comment-13750638
]
Phabricator commented on HIVE-4002:
-----------------------------------
yhuai has commented on the revision "HIVE-4002 [jira] Fetch task aggregation
for simple group by query".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:3631 Seems
that this line is the same as the line 3633
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6985 Why do
we need to change getInternalName to field? If we want to use field instead of
getInternalName, can you also make this to other places of this class?
ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java:582 why do we need
flushOp? I think it is not necessary to have flushOp. Also, can you change "an
blocking operator" to "a blocking operator"? I am sorry about the typo I made...
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java:493 I think we
can just use "operator.flush()" to tell GBY to process its buffer.
REVISION DETAIL
https://reviews.facebook.net/D8739
To: JIRA, navis
Cc: yhuai
> Fetch task aggregation for simple group by query
> ------------------------------------------------
>
> Key: HIVE-4002
> URL: https://issues.apache.org/jira/browse/HIVE-4002
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Minor
> Attachments: HIVE-4002.D8739.1.patch, HIVE-4002.D8739.2.patch,
> HIVE-4002.D8739.3.patch
>
>
> Aggregation queries with no group-by clause (for example, select count(*)
> from src) executes final aggregation in single reduce task. But it's too
> small even for single reducer because the most of UDAF generates just single
> row for map aggregation. If final fetch task can aggregate outputs from map
> tasks, shuffling time can be removed.
> This optimization transforms operator tree something like,
> TS-FIL-SEL-GBY1-RS-GBY2-SEL-FS + FETCH-TASK
> into
> TS-FIL-SEL-GBY1-FS + FETCH-TASK(GBY2-SEL-LS)
> With the patch, time taken for auto_join_filters.q test reduced to 6 min (10
> min, before).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
