[
https://issues.apache.org/jira/browse/SPARK-26269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715447#comment-16715447
]
ASF GitHub Bot commented on SPARK-26269:
----------------------------------------
tgravescs commented on issue #23223: [SPARK-26269][YARN]Yarnallocator should
have same blacklist behaviour with yarn to maxmize use of cluster resource
URL: https://github.com/apache/spark/pull/23223#issuecomment-445943499
ok thanks for trying. if I get a chance I can try later in the week, but
that doesn't have to block this now if someone else has time to review before I
get to it. We can always pull it back later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> YarnAllocator should have same blacklist behaviour with YARN to maxmize use
> of cluster resource
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-26269
> URL: https://issues.apache.org/jira/browse/SPARK-26269
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 2.3.1, 2.3.2, 2.4.0
> Reporter: wuyi
> Priority: Minor
> Fix For: 2.4.0
>
>
> Currently, YarnAllocator may put a node with a completed container whose exit
> status is not one of SUCCESS, PREEMPTED, KILLED_EXCEEDED_VMEM,
> KILLED_EXCEEDED_PMEM into blacklist. Howerver, for other exit status, e.g.
> KILLED_BY_RESOURCEMANAGER, Yarn do not consider its related nodes shoule be
> added into blacklist(see YARN's explaination for detail
> https://github.com/apache/hadoop/blob/228156cfd1b474988bc4fedfbf7edddc87db41e3/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java#L273).
> So, relaxing the current blacklist rule and having the same blacklist
> behaviour with YARN would maxmize use of cluster resources.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
