[
https://issues.apache.org/jira/browse/HIVE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811501#comment-13811501
]
Sergey Shelukhin commented on HIVE-5705:
----------------------------------------
[~hagleitn] fyi this is the jira we were talking about yday
> TopN might use better heuristic for disable
> -------------------------------------------
>
> Key: HIVE-5705
> URL: https://issues.apache.org/jira/browse/HIVE-5705
> Project: Hive
> Issue Type: Improvement
> Reporter: Sergey Shelukhin
> Priority: Minor
>
> Right now, if TopN overruns memory threshold it disables itself if it
> couldn't directly exclude rows as they are sent; it doesn't count evictions
> that were initially put in the heap and then superceded for this purpose.
> It's reasonable in most cases, but if N is relatively small, and map output
> is large, the cost could still be worth it even if rows don't get excluded
> immediately and are only evicted after being stored for some time. So we'd
> pay some memory copies but emit much less rows.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
