[
https://issues.apache.org/jira/browse/YARN-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268877#comment-16268877
]
Arun Suresh commented on YARN-7522:
-----------------------------------
Thanks for working on this [~wangda]..
In general, the idea looks fine. Couple of things to consider while fleshing
this out:
* We need to figure out at which point in the scheduler / container life cycle
are we planning on calling the addContainer and removeContainer. I propose we
do so in a scheduler agnostic manner - Somewhere in the AbstractYarnScheduler /
the AppSchedulingInfo, at the point of allocation (unfortunately, I don't think
AppSchdulingInfo is notified of container release/removal) or the
SchedulingAppAttempt.
* How we are planing on persisting this across RM restarts ? - I am not in
favor of pushing all this information into ZK. This unfortunately means, the
tags have to be pushed down to the NM so they can be retrieved from the NM
heartbeats during RM recovery. In that case, we have to figure out how to deal
with the delay from ACQUIRED to RUNNING of a container. The former is when the
RM has allocated and notified the AM and the later is when the NM actually gets
to know about the Container (after the AM has called start container). If we
are relying on the NM to persist this information, we should update the Tag
manager only after the the NM notifies the RM of the running container.
> Add application tags manager implementation
> -------------------------------------------
>
> Key: YARN-7522
> URL: https://issues.apache.org/jira/browse/YARN-7522
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-7522.YARN-6592.wip-001.patch
>
>
> This is different from YARN-6596, YARN-6596 is targeted to add constraint
> manager to store intra/inter application placement constraints. This JIRA is
> targeted to support storing maps between container-tags/applications and
> nodes. This will be required by affinity/anti-affinity implementation and
> cardinality.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
