[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480403#comment-16480403
]
Sunil Govindan commented on YARN-7494:
--------------------------------------
Thanks [~cheersyang] and [~leftnoteasy]
Attaching latest patch addressing comments. Pls find some clarifications for
few comments.
bq.In another word, please make sure {{getNodesPerPartition}} returns all nodes
if there is no partition configured.
Yes. I tested this locally. Also added test case to ensure that nodes are
returned in normal cluster setup case and also when labels are configured.
bq.you can't call getCSLeafQueue because it assumes it is CS scheduler
I added to Queue base class. This comes correctly now and wont cause any test
issue. Its better to handle this way otherwise we have to write few apis to
achieve same from CS to app
bq.line 57: {{#getPreferredNodeIterator}} this API should not depend on
{{Collection<N> nodes}}, can you please double check.
Default policy will accept nodes from CS and sort it on demand.
ResourceUsageBased will fetch from Sorter thread. Wangda also suggested to keep
both. Hence i added this api to handle both scenarios
bq.what's the difference of this class to DefaultMultiNodeLookupPolicy
As I mentioned above, DefaultMultiNodeLookupPolicy wont take nodes from Sorted
thread rather it will do sort on demand. This is a choice to user and enabled
as needed.
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7494.001.patch, YARN-7494.002.patch,
> YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch,
> YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch,
> YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
