zhihai xu created YARN-4133: ------------------------------- Summary: Containers to be preempted leaks in FairScheduler preemption logic. Key: YARN-4133 URL: https://issues.apache.org/jira/browse/YARN-4133 Project: Hadoop YARN Issue Type: Bug Components: fairscheduler Affects Versions: 2.7.1 Reporter: zhihai xu Assignee: zhihai xu
Containers to be preempted leaks in FairScheduler preemption logic. It may cause missing preemption due to containers in {{warnedContainers}} wrongly removed. The problem is in {{preemptResources}}: There are two issues which can cause containers wrongly removed from {{warnedContainers}}: Firstly missing the container state {{RMContainerState.ACQUIRED}} in the condition check: {code} (container.getState() == RMContainerState.RUNNING || container.getState() == RMContainerState.ALLOCATED) {code} Secondly if {{isResourceGreaterThanNone(toPreempt)}} return false, we shouldn't remove container from {{warnedContainers}}, We should only remove container from {{warnedContainers}}, if container is not in state {{RMContainerState.RUNNING}}, {{RMContainerState.ALLOCATED}} and {{RMContainerState.ACQUIRED}}. {code} if ((container.getState() == RMContainerState.RUNNING || container.getState() == RMContainerState.ALLOCATED) && isResourceGreaterThanNone(toPreempt)) { warnOrKillContainer(container); Resources.subtractFrom(toPreempt, container.getContainer().getResource()); } else { warnedIter.remove(); } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)