[
https://issues.apache.org/jira/browse/FLINK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15882320#comment-15882320
]
ASF GitHub Bot commented on FLINK-5791:
---------------------------------------
Github user shuai-xu commented on a diff in the pull request:
https://github.com/apache/flink/pull/3304#discussion_r102907061
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java ---
@@ -376,4 +388,23 @@ private int generatePriority(ResourceProfile
resourceProfile) {
}
}
+ /**
+ * Get resource profile by given the container priority. It should
always be got from the mapping
+ * between resource profile and priority, otherwise it will throw the
exception.
+ *
+ * @param priority The container priority used for distinguishing
different resources.
+ * @return The resource profile corresponding with the priority.
+ * @throws Exception
+ */
+ private ResourceProfile getResourceProfile(int priority) throws
Exception{
+ if (resourcePriorities.containsValue(priority)) {
+ for (Map.Entry<ResourceProfile, Integer> entry :
resourcePriorities.entrySet()) {
+ if (entry.getValue() == priority) {
+ return entry.getKey();
+ }
+ }
+ }
--- End diff --
This is a confusing part in yarn. Now the priority is only used to
distinguish requests of different resource. It has nothing to do with
importance. For slot request of different resources, we should generate
different priorities for them, and put priority and resource profile to the
resourcePriorities map. When yarn containers come back, we know they should
start the task manager with which resource profile by query the map with the
container priority. And when task manager offer slot with the resource profile
to job master, we can exactly match the slot with the original request.
> Resource should be strictly matched when allocating for yarn
> ------------------------------------------------------------
>
> Key: FLINK-5791
> URL: https://issues.apache.org/jira/browse/FLINK-5791
> Project: Flink
> Issue Type: Improvement
> Components: YARN
> Reporter: shuai.xu
> Assignee: shuai.xu
> Labels: flip-6
>
> In flip6, for yarn mode, resource should be assigned as requested to avoid
> resource wasting and OOM.
> 1. YarnResourceManager will request container according to ResourceProfile
> in slot request form JM.
> 2. RM will pass the ResourceProfile to TM for initializing its slots.
> 3. RM should match the slots offered by TM with SlotRequest from JM strictly.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
