[ https://issues.apache.org/jira/browse/IMPALA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sahil Takiar resolved IMPALA-8780. ---------------------------------- Resolution: Fixed Fix Version/s: Impala 3.3.0 Closing as fixed. We ended up referring the complete re-factoring of IMPALA-8779 to a later patch. > Implementation of BufferedPlanRootSink where FlushFinal blocks until all rows > are fetched > ----------------------------------------------------------------------------------------- > > Key: IMPALA-8780 > URL: https://issues.apache.org/jira/browse/IMPALA-8780 > Project: IMPALA > Issue Type: Sub-task > Components: Backend > Reporter: Sahil Takiar > Assignee: Sahil Takiar > Priority: Major > Fix For: Impala 3.3.0 > > > Implement {{BufferedPlanRootSink}} so that {{FlushFinal}} blocks until all > rows are fetched. The implementation should use the {{RowBatchQueue}} > introduced by IMPALA-8779. By blocking in {{FlushFinal}} all non-coordinator > fragments will be closed if all results fit in the {{RowBatchQueue}}. > {{BufferedPlanRootSink::Send}} should enqueue each given {{RowBatch}} onto > the queue and then return. If the queue is full, it should block until there > is more space left in the queue. {{BufferedPlanRootSink::GetNext}} reads from > the queue and then fills in the given {{QueryResultSet}} by using the > {{DataSink}} {{ScalarExprEvaluator}}-s. Since the producer thread can call > {{BufferedPlanRootSink::Close}} while the consumer is calling > {{BufferedPlanRootSink::GetNext}} the two methods need to be synchronized so > that the {{DataSink}} {{MemTracker}}-s are not closed while {{GetNext}} is > running. > The implementation of {{BufferedPlanRootSink}} should remain the same > regardless of whether a {{std::queue}} backed {{RowBatchQueue}} or a > {{BufferedTupleStream}} backed {{RowBatchQueue}} is used. > {{BufferedPlanRootSink}} and {{BlockingPlanRootSink}} are similar in the > sense that {{BlockingPlanRootSink}} buffers one {{RowBatch}}, so for queries > that return under 1024 rows, all non-coordinator fragments are closed > immediately as well. The advantage of {{BufferedPlanRootSink}} is that allows > buffering for 1+ {{RowBatch}}-es. -- This message was sent by Atlassian JIRA (v7.6.14#76016)