[
https://issues.apache.org/jira/browse/DRILL-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098449#comment-15098449
]
Jacques Nadeau commented on DRILL-4236:
---------------------------------------
This is a separate bug we should file. The external sort operation needs to
reserve any necessary memory and ensure that it doesn't ever hit an out of
memory condition.
[~adeneche] & [~sphillips], it seems like we need to go through with a fine
tooth comb and ensure that all possible memory allocations within the External
sort are correctly handled. It may make sense to break into two possible
situations: easily accommodated and hard to accommodate. For the latter, we
probably need to create a separate child allocator with a reservation and usage
policy that ensures we don't hit out of memory.
> ExternalSort should use the new allocator functionality to better manage it's
> memory usage
> ------------------------------------------------------------------------------------------
>
> Key: DRILL-4236
> URL: https://issues.apache.org/jira/browse/DRILL-4236
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.5.0
> Reporter: Deneche A. Hakim
> Assignee: Deneche A. Hakim
> Labels: documentation
> Fix For: 1.5.0
>
>
> Before DRILL-4215, the sort operator wasn't able to correctly compute it's
> memory usage, and so it jumped through a bunch of hoops to try to figure out
> when it should spill to disk.
> With the transfer accounting in place, this code can be greatly simplified to
> just use the current operator memory allocation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
