[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564589#comment-16564589
]
Wangda Tan commented on YARN-7494:
----------------------------------
[~sunilg],
Thanks for updating the patch, some comments.
1) Not used:
public static final String RESOURCE_USAGE_BASED_NODE_SORTING_POLICY =
"resource-usage";
2) Renames for CapacitySchedulerConfiguration:
- DEFAULT_NODE_SORTING_POLICY_NAME => DEFAULT_NODE_SORTING_POLICY_CLASSNAME
- For the usage, MULTI_NODE_SORTING_POLICIES/MULTI_NODE_SORTING_POLICY_NAME
could be confusion to end users. Is it better to have same prefix, such as:
multi-node-sorting.policy.classes = "a,b,c"
multi-node-sorting.policy.a.class = "org.apache..."
"classes" will be a disallowed policy name.
3) CS#getCandidateNodeSet copies the full list everytime, is there any plan to
improve this in the future?
4) DefaultMultiNodeLookupPolicy: Should we call it
ResourceUsageMultiNodeLookupPolicy (or LeastResourceUsedPreferred...?
5) It's better to add getMultiNodeSortingPolicyName CSQueue, inside
FiCaSchedulerApp, you can check the instanceof. With this we can limit the
changes to CS.
6) RegularContainerAllocator#allocate, when the change needed?
7) Instead of adding MultiNodeSortingManager to RMContext, can we limit changes
inside Scheduler?
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7494.001.patch, YARN-7494.002.patch,
> YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch,
> YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch,
> YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch,
> YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
