[jira] [Resolved] (IMPALA-8803) Coordinator should release admitted memory per-backend rather than per-query
[
https://issues.apache.org/jira/browse/IMPALA-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-8803.
--
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> Coordinator should release admitted memory per-backend rather than per-query
>
>
> Key: IMPALA-8803
> URL: https://issues.apache.org/jira/browse/IMPALA-8803
> Project: IMPALA
> Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: Impala 3.4.0
>
>
> When {{SPOOL_QUERY_RESULTS}} is true, the coordinator backend may be long
> lived, even though all other backends for the query have completed.
> Currently, the Coordinator only releases admitted memory when the entire
> query has completed (include the coordinator fragment) -
> https://github.com/apache/impala/blob/72c9370856d7436885adbee3e8da7e7d9336df15/be/src/runtime/coordinator.cc#L562
> In order to more aggressively return admitted memory, the coordinator should
> release memory when each backend for a query completes, rather than waiting
> for the entire query to complete.
> Releasing memory per backend should be batched because releasing admitted
> memory in the admission controller requires obtaining a global lock and
> refreshing the internal stats of the admission controller. Batching will help
> mitigate any additional overhead from releasing admitted memory per backend.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (IMPALA-8803) Coordinator should release admitted memory per-backend rather than per-query
[
https://issues.apache.org/jira/browse/IMPALA-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-8803.
--
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> Coordinator should release admitted memory per-backend rather than per-query
>
>
> Key: IMPALA-8803
> URL: https://issues.apache.org/jira/browse/IMPALA-8803
> Project: IMPALA
> Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: Impala 3.4.0
>
>
> When {{SPOOL_QUERY_RESULTS}} is true, the coordinator backend may be long
> lived, even though all other backends for the query have completed.
> Currently, the Coordinator only releases admitted memory when the entire
> query has completed (include the coordinator fragment) -
> https://github.com/apache/impala/blob/72c9370856d7436885adbee3e8da7e7d9336df15/be/src/runtime/coordinator.cc#L562
> In order to more aggressively return admitted memory, the coordinator should
> release memory when each backend for a query completes, rather than waiting
> for the entire query to complete.
> Releasing memory per backend should be batched because releasing admitted
> memory in the admission controller requires obtaining a global lock and
> refreshing the internal stats of the admission controller. Batching will help
> mitigate any additional overhead from releasing admitted memory per backend.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
