[ https://issues.apache.org/jira/browse/YARN-9233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769167#comment-16769167 ]
Bilwa S T edited comment on YARN-9233 at 2/15/19 10:23 AM: ----------------------------------------------------------- Thanks [~rohithsharma] for reviewing {quote}This is Spark AM issue. I think skipping event is a better solution. I have attached a patch for it. Please review {quote} was (Author: bilwast): Thanks [~rohithsharma] for reviewing {quote} This is Spark AM issue. I think skipping event is a better solution. I have attached a patch for it. Please review {quote} > RM may report allocated container which is killed (but not acquired by AM ) > to AM which can cause spark AM confused > ------------------------------------------------------------------------------------------------------------------- > > Key: YARN-9233 > URL: https://issues.apache.org/jira/browse/YARN-9233 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Bilwa S T > Assignee: Bilwa S T > Priority: Major > Attachments: YARN-9233-001.patch, YARN-9233-002.patch > > > After the RM kills an allocated (Allocated state) container for various > reasons, it will go through the state transition process to the FINISHED > state just like other state containers. Currently RM doesn't consider if > container is acquired by the AM. Hence All the containers transitioned to > FINISH state are added to justFinishedContainers list. Therefore the > container that is not obtained by the AM and is killed by the rm will also > return through the AM heartbeat. So AM re-applies for more resources than > needed which would eventually cause number of containers to exceed the > maximum limit -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org