[ https://issues.apache.org/jira/browse/DRILL-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pritesh Maker updated DRILL-6498: --------------------------------- Reviewer: Parth Chandra > Support for EMIT outcome in ExternalSortBatch > --------------------------------------------- > > Key: DRILL-6498 > URL: https://issues.apache.org/jira/browse/DRILL-6498 > Project: Apache Drill > Issue Type: Task > Components: Execution - Relational Operators > Reporter: Sorabh Hamirwasia > Assignee: Sorabh Hamirwasia > Priority: Major > Fix For: 1.14.0 > > > With Lateral and Unnest if Sort is present in the sub-query, then it needs to > handle the EMIT outcome correctly. This means when a EMIT is received then > perform the Sort operation on the records buffered so far and produce output > with it. After EMIT Sort should refresh it's state and again work on next > batches of incoming record unless an EMIT is seen again. > For first cut Sort will not support spilling in the subquery between Lateral > and Unnest since spilling is very unlikely. The worst case that can happen is > that Lateral will get a batch with only 1 row of data because of repeated > type column data size being too big. In that case Unnest will produce 1 > output batch only and Sort or other blocking operators anyways needs enough > memory to at least hold 1 incoming batch. So in ideal cases spilling should > not happen. But if there is a operator between Sort and Unnest which > increases the data size then Sort might be in a situation to spill but thats > not a common case for now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)