[
https://issues.apache.org/jira/browse/IMPALA-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18073519#comment-18073519
]
ASF subversion and git services commented on IMPALA-14840:
----------------------------------------------------------
Commit 6cbab68914cf536ade27db635f8aa34f2cb43428 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6cbab6891 ]
IMPALA-14840: Enforce pool Max Memory when query limits are unset
Impala should use memory-based query admission when Max Memory is set
for the given pool. However, if Min Query Memory Limit, Max Query Memory
Limit are missing, and query option MEM_LIMIT is not set, then currently
queries can run with unlimited resources.
This can cause a situation when Max Memory is specified, but the pool
can consume basically unlimited memory.
This patch fixes ScheduleState::UpdateMemoryRequirements() to not turn
off memory-based query admission when Max Memory is set.
The patch also removes the unused 'root_cfg' parameter of
AdmissionController::CanAdmitRequest().
Testing
* new admission controller backend tests added
* updated pool 'poolNoMemLimits' in mem-limit-test-fair-scheduler.xml
to not specify Max Memory
Change-Id: I4834964e4361895e10627a661831253ce676c129
Reviewed-on: http://gerrit.cloudera.org:8080/24129
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Pool can use more memory than Max Memory when Min/Max Query Memory limit is
> not set
> -----------------------------------------------------------------------------------
>
> Key: IMPALA-14840
> URL: https://issues.apache.org/jira/browse/IMPALA-14840
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> Based on the following comments a query can use unlimited resources if
> * Pool has no Min and Max Query Memory LImit
> * Query option MEM_LIMIT is not set
> https://github.com/apache/impala/blob/787b63a4b88b91bf441200e336ca0bac873541b9/be/src/scheduling/schedule-state.cc#L301-L303
> {noformat}
> // If the min_query_mem_limit and max_query_mem_limit are not set in the
> pool config
> // then it falls back to traditional(old) behavior, which means that, it
> sets the
> // mem_limit if it is set in the query options, else sets it to -1 (no
> limit).
> {noformat}
> https://github.com/apache/impala/blob/787b63a4b88b91bf441200e336ca0bac873541b9/be/src/scheduling/schedule-state.cc#L411
> This can lead to using more memory than the specified "Max Memory" for the
> pool.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
