[ https://issues.apache.org/jira/browse/IMPALA-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Rodoni reopened IMPALA-7836: --------------------------------- > Impala 3.1 Doc: New query option 'topn_bytes_limit' for TopN to Sort > conversion > ------------------------------------------------------------------------------- > > Key: IMPALA-7836 > URL: https://issues.apache.org/jira/browse/IMPALA-7836 > Project: IMPALA > Issue Type: Sub-task > Components: Docs, Frontend > Affects Versions: Impala 2.9.0 > Reporter: Sahil Takiar > Assignee: Alex Rodoni > Priority: Major > Labels: future_release_doc > Fix For: Impala 3.1.0 > > > IMPALA-5004 adds a new query level option called 'topn_bytes_limit' that we > should document. The changes in IMPALA-5004 work by estimating the amount of > memory required to run a TopN operator. The memory estimate is based on the > size of the individual tuples that need to be processed by the TopN operator, > as well as the sum of the limit and offset in the query. TopN operators don't > spill to disk so they have to keep all rows they process in memory. > If the estimated size of the working set of the TopN operator exceeds the > threshold of 'topn_bytes_limit' the TopN operator will be replaced with a > Sort operator. The Sort operator can spill to disk, but it processes all the > data (the limit and offset have no affect). So switching to Sort might incur > performance penalties, but it will require less memory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org