[ https://issues.apache.org/jira/browse/YARN-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317379#comment-16317379 ]
Arun Suresh commented on YARN-6599: ----------------------------------- [~leftnoteasy], did a slightly deeper dive. This is my understanding of the general flow - I'd break this down the 2 phases. # When the app makes the allocate call: This is where you do the updatePendingAsk etc for each SchedulingRequest. This is also where you instantiate the AppPlacementAllocator and map it to the request. Looks like the really interesting method is {{validateAndSetSchedulingRequest}}, which is where you validate the placement constraints. This method sets the valid targetAllocationTags etc. # When the node heartbeats: At this point, in the leafQueue, we pick the SchedulingRequest with highest priority and finally, we call the {{canAllocate}} method which checks if the Node can be assigned to the scheduling request based on the placementConstratint. right ? Given the above, we should agree that: This approach - while it ensures that allocation of a SchedulingRequest to a node will guarantee that Constraints are NOT violated, It does nothing in the way of trying to FIND the nodes that will meet the constraints.. agreed ? My opinion - and something we should call out / document is that: * If an app is more concerned about priority ordering of its requests, then we can recommend using this approach. * If the app the more concerned about an optimal placement, then the processor route will give better results - since it activly tries to find nodes that satisfy constraints by considering multiple schedulingRequests and multiple nodes. Thuoghts ? Also some other comments: * In the Scheduler, you are adding a new List<SchedulingRequest> parameter to the allocate() method. Do you think, a better approach might be to create a common superclass / interface which both the SchedulingRequest and ResourceRequest extends and change the existing parameter to {{List<? extends BaseRequest>}} ? * If we do the above, then you wont have to duplicate methods like : application.updateResourceRequests and normalizeSchedulingRequests > Support rich placement constraints in scheduler > ----------------------------------------------- > > Key: YARN-6599 > URL: https://issues.apache.org/jira/browse/YARN-6599 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Wangda Tan > Assignee: Wangda Tan > Attachments: YARN-6599-YARN-6592.003.patch, > YARN-6599-YARN-6592.004.patch, YARN-6599-YARN-6592.005.patch, > YARN-6599-YARN-6592.006.patch, YARN-6599-YARN-6592.007.patch, > YARN-6599-YARN-6592.008.patch, YARN-6599-YARN-6592.wip.002.patch, > YARN-6599.poc.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org