[ https://issues.apache.org/jira/browse/YARN-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14966595#comment-14966595 ]
Sunil G commented on YARN-4284: ------------------------------- Yes [~steve_l], clusterwise blacklisting is a potential problem. YARN-2005 has a config parameter named {{yarn.am.blacklisting.disable-failure-threshold}} with a default value of 0.8. This means that one application can only blacklist a maximum of 80% of nodes in cluster. This is to ensure that some nodes has to be selected. > condition for AM blacklisting is too narrow > ------------------------------------------- > > Key: YARN-4284 > URL: https://issues.apache.org/jira/browse/YARN-4284 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager > Affects Versions: 2.8.0 > Reporter: Sangjin Lee > Assignee: Sangjin Lee > Attachments: YARN-4284.001.patch > > > Per YARN-2005, there is now a way to blacklist nodes for AM purposes so the > next app attempt can be assigned to a different node. > However, currently the condition under which the node gets blacklisted is > limited to {{DISKS_FAILED}}. There are a whole host of other issues that may > cause the failure, for which we want to locate the AM elsewhere; e.g. disks > full, JVM crashes, memory issues, etc. > Since the AM blacklisting is per-app, there is little practical downside in > blacklisting the nodes on *any failure* (although it might lead to > blacklisting the node more aggressively than necessary). I would propose > locating the next app attempt to a different node on any failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)