[ https://issues.apache.org/jira/browse/YARN-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16804580#comment-16804580 ]
Hadoop QA commented on YARN-9413: --------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color} | {color:red} YARN-9413 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | YARN-9413 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/23834/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Queue resource leak after app fail for CapacityScheduler > -------------------------------------------------------- > > Key: YARN-9413 > URL: https://issues.apache.org/jira/browse/YARN-9413 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler > Affects Versions: 3.1.2 > Reporter: Tao Yang > Assignee: Tao Yang > Priority: Major > Attachments: YARN-9413.001.patch, YARN-9413.002.patch, > image-2019-03-29-10-47-47-953.png > > > To reproduce this problem: > # Submit an app which is configured to keep containers across app attempts > and should fail after AM finished at first time (am-max-attempts=1). > # App is started with 2 containers running on NM1 node. > # Fail the AM of the application with PREEMPTED exit status which should not > count towards max attempt retry but app will fail immediately. > # Used resource of this queue leaks after app fail. > The root cause is the inconsistency of handling app attempt failure between > RMAppAttemptImpl$BaseFinalTransition#transition and > RMAppImpl$AttemptFailedTransition#transition: > # After app fail, RMAppFailedAttemptEvent will be sent in > RMAppAttemptImpl$BaseFinalTransition#transition, if exit status of AM > container is PREEMPTED/ABORTED/DISKS_FAILED/KILLED_BY_RESOURCEMANAGER, it > will not count towards max attempt retry, so that it will send > AppAttemptRemovedSchedulerEvent with keepContainersAcrossAppAttempts=true and > RMAppFailedAttemptEvent with transferStateFromPreviousAttempt=true. > # RMAppImpl$AttemptFailedTransition#transition handle > RMAppFailedAttemptEvent and will fail the app if its max app attempts is 1. > # CapacityScheduler handles AppAttemptRemovedSchedulerEvent in > CapcityScheduler#doneApplicationAttempt, it will skip killing and calling > completion process for containers belong to this app, so that queue resource > leak happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org