[
https://issues.apache.org/jira/browse/TEZ-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000006#comment-14000006
]
Siddharth Seth commented on TEZ-1039:
-------------------------------------
Couple of comments
- canFit - Does this belong in containerSignatureMatcher, which has other
similar functionality of matching container resources etc.
- Looks like quite a bit of code between the two allocate methods can be
collapsed together. The only difference I see is the extra containerId in the
CookieContainerRequest construction.
- getMatchingRequestWithoutPriority - If I'm reading this right, we'll end up
scanning all tasks at a priority level for a container, even if none of them
requested container affinity. That's very expensive. Instead, can this be
changed to scan the list only if there are affinitized containers, and
potentially walk only the list of affinitized containers while doing the task
level match. Another method in TezAMRMClientAsync siimlar to
getMatchingRequestsForTopPriority - which returns only affinitized containers
for that priority.
- Does getMatchingRequestWithPriority also need to change ? The case being
containerAffinity within the same vertex. This can always be deferred to when
it's needed though.
- In the unit test, a couple of containers are mocked - may be easier to just
use the createContainer method which already exists in the unit tests. Haven't
looked at the rest of the unit test in detail.
> Add Container locality to TaskScheduler
> ---------------------------------------
>
> Key: TEZ-1039
> URL: https://issues.apache.org/jira/browse/TEZ-1039
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1039.1.patch
>
>
> Currently, the task scheduler support node and rack locality. Proposal is to
> add another level - container locality (given a container) which means first
> try to assign to the container, then fall back to the container's node, rack
> and so on.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
