[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16345090#comment-16345090
]
Weiwei Yang commented on YARN-7494:
-----------------------------------
Hi [~leftnoteasy], [~sunilg]
I have some high level comments (apologies I did not look into the details in
this patch yet). Since this is one of the most critical part of the scheduler
improvement, lets make sure the design is thoughtful. My comments below,
# From the API level, I think we should have the an ordered list of nodes
per-algorithm (or the ordering policy) instead of per-app, because the sorting
is expensive. The approach in this patch creates an instance of
NodeUsageBasedPolicy and sorts the nodes every time which is costly to me.
Basically I am suggesting, user can specify app-level nodes ordering policy,
but if two apps are using same policy, they will access one same list.
# We should be able to apply placement constraints into the ordered nodes, as
filters. That can be used to filter out some illegal candidates which fails to
satisfy a given constraint. I am not sure where is the best place for it, but
suggest to think about this.
[~Tao Yang], please do also take a look.
Thanks
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Priority: Major
> Attachments: YARN-7494.001.patch, YARN-7494.v0.patch,
> YARN-7494.v1.patch
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
