[
https://issues.apache.org/jira/browse/IMPALA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-691.
----------------------------------
Resolution: Fixed
https://gerrit.cloudera.org/#/c/12262/ actually fixed this for containers
already by changing the default to include the JVM memory.
> Process mem limit does not account for the JVM's memory usage
> -------------------------------------------------------------
>
> Key: IMPALA-691
> URL: https://issues.apache.org/jira/browse/IMPALA-691
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 1.2.1, Impala 2.0, Impala 2.1, Impala 2.2, Impala
> 2.3.0
> Reporter: Skye Wanderman-Milne
> Assignee: Tim Armstrong
> Priority: Major
> Labels: incompatibility, resource-management
>
> The JVM doesn't appear to use malloc, so it's memory usage is not reported by
> tcmalloc and we do not count it in the process mem limit. I verified this by
> adding a large allocation in the FE, and noting that the total memory usage
> (virtual or resident) reported in /memz is not affected, but the virtual and
> resident memory usage reported by top is.
> This is problematic especially because Impala caches table metadata in the FE
> (JVM) which can become quite big (few GBs) in extreme cases.
> *Workaround*
> As a workaround, we recommend reducing the process memory limit by 1-2GB to
> "reserve" memory for the JVM. How much memory you should reserve typically
> depends on the size of your catalog ( number of
> tables/partitions/columns/blocks etc.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
