[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802763#comment-17802763 ] Shilun Fan commented on YARN-4636: -- Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203367#comment-15203367 ] Sunil G commented on YARN-4636: --- As YARN improves in its blacklist/whitelist node functionality, one of the major usecase from our end is to save the second/further AM Container launch attempts to same failed node (if this is failed in a node due to external environment/memory issues). This can really help us. With YARN-2005, we have a mechanism in hand. And there were concerns on its strict behavior. Proposal made in YARN-4837 helps in straighten things out for immediate 2.8. I think YARN-4576 was trying to improve on current YARN-2005 and trying to generalize it. As we are going forward, if we are planning for a global blacklisting based various type of container exit codes, then policy can be helpful assuming that we may have different type of apps. For this scenario, we do not have usecases from our end. I checked with [~rohithsharma] and [~Naganarasimha Garla] also for this. It will be good if we can discuss/retrospect more on *global blacklisting* and its advantages/limitations based on current available information from containers exit codes. > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201206#comment-15201206 ] Vinod Kumar Vavilapalli commented on YARN-4636: --- -1 for something like this without understanding the use-cases. IMO, the "AM blacklisting" doesn't even need to be user-visible (YARN-4837) let alone be pluggable. > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201373#comment-15201373 ] Junping Du commented on YARN-4636: -- bq. -1 for something like this without understanding the use-cases. We should ask for user cases first before making -1 decision. bq. IMO, the "AM blacklisting" doesn't even need to be user-visible (YARN-4837) let alone be pluggable. Pluggable blacklist policy is necessary because application's requirement for AM robust is different. Some app can tolerant AM failure (small and short-running job) but some apps don't want any risk (like a large MR job with long running reducer jobs - AM restart will kill reducer tasks no mater how long it is already running). IMO, Allowing various blacklist policies is a good thing for YARN to show the extension capability to address different application's requirement especially for a cluster form of heterogeneous nodes. Any comments from guys in watching list? > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115557#comment-15115557 ] Sunil G commented on YARN-4636: --- Yes. That's correct. I will also help in reviewing YARN-4635. ThankYou. > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115539#comment-15115539 ] Sunil G commented on YARN-4636: --- Hi [~djp] I could help to get this in. Pls let me know if you have plans. :) > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4636) Make blacklist tracking policy pluggable for more extensions.
[ https://issues.apache.org/jira/browse/YARN-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115553#comment-15115553 ] Junping Du commented on YARN-4636: -- Sure. Please feel free to take it. However, I would suggest to start the work here after YARN-4635 or it will get rebased multiple times. > Make blacklist tracking policy pluggable for more extensions. > - > > Key: YARN-4636 > URL: https://issues.apache.org/jira/browse/YARN-4636 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Sunil G > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
