[ https://issues.apache.org/jira/browse/DRILL-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Rogers reassigned DRILL-4181: ---------------------------------- Assignee: Paul Rogers > Improve memory limit assignment for Sort operators > -------------------------------------------------- > > Key: DRILL-4181 > URL: https://issues.apache.org/jira/browse/DRILL-4181 > Project: Apache Drill > Issue Type: Improvement > Affects Versions: 1.4.0 > Reporter: Aman Sinha > Assignee: Paul Rogers > > Currently, the max_query_memory_per_node is divided among all the Sort > operators in the query plan, regardless of whether the Sort operators are > executing at the same time. Since Sort is a blocking operator, the number of > concurrent Sorts running is limited. Thus, the Sort on both sides of a > MergeJoin could be concurrently executing but a Sort that occurs above the > MergeJoin cannot start until the MergeJoin produces a row which is only when > both the child Sorts have produced a row. > Due to the conservative estimate, we have seen queries such as TPC-H Q8 with > forced MergeJoin run out-of-memory even with high max_query_memory_per_node. > This query plan has 15 Sort operators and with max_width = 23, > max_query_memory_per_node = 20GB, each Sort gets only 63MB (20GB/(23*15)). > We should improve the algorithm for computing the memory limit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)