[
https://issues.apache.org/jira/browse/SAMZA-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Pan (Data Infrastructure) updated SAMZA-972:
-----------------------------------------------
Fix Version/s: 0.11.0
> Add holistic memory monitoring for SamzaContainer
> -------------------------------------------------
>
> Key: SAMZA-972
> URL: https://issues.apache.org/jira/browse/SAMZA-972
> Project: Samza
> Issue Type: Bug
> Reporter: Jagadish
> Assignee: Jagadish
> Fix For: 0.11.0
>
> Attachments: SAMZA-972-1.patch
>
>
> This feature introduces physical memory monitoring in SamzaContainer.
> Context:
> Often memory used by the SamzaContainer process includes
> A. JVM Heap memory: This is where all JVM variables live.
> B. Native memory: This memory lives out of the JVM heap and is not visible to
> the JVM. Examples include used by RocksDb, native libraries that user code
> depends on etc.
> User jobs could be killed by Yarn if their total memory (A+B) exceeds the
> configured maximum of yarn.container.memory.mb.
> Currently, while our existing metrics provide visibility into [A] via JMX, we
> don't have visibility into [B]. (as it's totally external to the JVM).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
