[
https://issues.apache.org/jira/browse/YARN-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002273#comment-17002273
]
zhoukang commented on YARN-10047:
---------------------------------
[~wilfreds]thanks for your reply, for some case the memory consume will
consider subprocess which will has incorrect memory usage.
> Memory consume of process tree will consider subprocess which may make
> container exit unexcepted
> -------------------------------------------------------------------------------------------------
>
> Key: YARN-10047
> URL: https://issues.apache.org/jira/browse/YARN-10047
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: zhoukang
> Assignee: zhoukang
> Priority: Major
>
> As below, we have a case which spark driver execute some scripts.Then
> sometimes the driver will be killed.
> {code:java}
> yarn.174410.log.2019-12-17.000002:2019-12-17,06:59:14,831 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
> Container
> [pid=50529,containerID=container_e917_1576303656075_174957_01_003197] is
> running beyond physical memory limits. Current usage: 50.28 GB of 5.25 GB
> physical memory used; xxx. Killing container.
> {code}
> {code:java}
> boolean isProcessTreeOverLimit(String containerId,
> long currentMemUsage,
> long curMemUsageOfAgedProcesses,
> long vmemLimit) {
> boolean isOverLimit = false;
>
> /**
> if (currentMemUsage > (2 * vmemLimit)) {
> LOG.warn("Process tree for container: " + containerId
> + " running over twice " + "the configured limit. Limit=" +
> vmemLimit
> + ", current usage = " + currentMemUsage);
> isOverLimit = true;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
