[
https://issues.apache.org/jira/browse/IMPALA-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-8656.
----------------------------------
Fix Version/s: Impala 3.4.0
Resolution: Fixed
Closing this and marking as Fixed. All the dev work has been completed, and the
feature has been sitting in master for a few weeks and seems to be stable.
> Support for eagerly fetching and spooling all query result rows
> ---------------------------------------------------------------
>
> Key: IMPALA-8656
> URL: https://issues.apache.org/jira/browse/IMPALA-8656
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.12.0, Impala 3.2.0
> Reporter: Michael Ho
> Assignee: Sahil Takiar
> Priority: Critical
> Fix For: Impala 3.4.0
>
>
> Impala's current interaction with clients is pulled-based: it relies on
> clients to fetch results to trigger the generation of more result row batches
> until all the result rows have been produced. If a client issues a query
> without fetching all the results, the query fragments will continue to
> consume the resources until the query hits is cancelled and unregistered for
> whatever reasons. This is undesirable as resources are held up by misbehaving
> clients and other queries may wait for extended period of time in admission
> control due to this.
> The high level idea for this JIRA is for Impala to have a mode in which
> result sets of queries are eagerly fetched and spooled somewhere (preferably
> some persistent storage). In this way, the cluster's resources are freed up
> once all result rows have been fetched and stored in the spooling location.
> Incoming client fetches can be returned from this spooled locations.
> cc'ing [~stakiar], [~twm378], [~joemcdonnell], [~lv]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
