[
https://issues.apache.org/jira/browse/HIVE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528572#comment-13528572
]
Phabricator commented on HIVE-3562:
-----------------------------------
navis has commented on the revision "HIVE-3562 [jira] Some limit can be pushed
down to map stage".
Is this right direction?
INLINE COMMENTS
conf/hive-default.xml.template:1407 It's like a convention. When this patch
is believed to be fully stable, it'll be so.
conf/hive-default.xml.template:1413 It's size of memory, not number of rows.
10M seemed to be conservative enough.
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java:378 I just
wanted to keep it compact as is possible because this can be used with map
aggregation. I'm thinking of interface which can be implemented by user and
configured to be used for this.
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java:383 In RS,
o2 is the key and cannot be null and also, making this class support null key
needs some more works. I just ignored that case but yes, some comments could be
useful at the least.
REVISION DETAIL
https://reviews.facebook.net/D5967
BRANCH
DPAL-1910
To: JIRA, tarball, navis
Cc: njain
> Some limit can be pushed down to map stage
> ------------------------------------------
>
> Key: HIVE-3562
> URL: https://issues.apache.org/jira/browse/HIVE-3562
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3562.D5967.1.patch, HIVE-3562.D5967.2.patch
>
>
> Queries with limit clause (with reasonable number), for example
> {noformat}
> select * from src order by key limit 10;
> {noformat}
> makes operator tree,
> TS-SEL-RS-EXT-LIMIT-FS
> But LIMIT can be partially calculated in RS, reducing size of shuffling.
> TS-SEL-RS(TOP-N)-EXT-LIMIT-FS
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
