[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622544#comment-13622544 ]
Bikas Saha commented on YARN-392: --------------------------------- I dont think its possible for location specific and non-location specific requests to live at the same priority. This is mainly because of the way current schedulers are implemented in the RM (grouped together and keyed by location and priority). Such requests have to be separated by priority and that may not be a bad thing IMO. I discussed this offline with [~vinodkv] and I would like to suggest and extension to the approach. Instead of a flag, how about specifying a time interval that tells the RM how long to wait before dropping locality. A time interval of infinite would be the same as a boolean flag and so this approach covers the other one. Additionally, it lets a large map to be more conservative about dropping locality over latency and a short job more aggressive. Currently the value of this interval comes from config and maps to the number of scheduling attempts missed by this request. This is done by keeping a count of node heartbeats. Given the number of nodes and heartbeat interval, the user specified time interval can easily be mapped to a count that matches the current implementation. So this will not be a perf hit nor a change in logic compared to existing code. Another thing to consider is allowing users to say I want to be scheduled only on these racks. Again, I dont think we can mix node-specfic and rack-specific scheduling at the same priority. > Make it possible to schedule to specific nodes without dropping locality > ------------------------------------------------------------------------ > > Key: YARN-392 > URL: https://issues.apache.org/jira/browse/YARN-392 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Bikas Saha > Assignee: Sandy Ryza > Attachments: YARN-392-1.patch, YARN-392.patch > > > Currently its not possible to specify scheduling requests for specific nodes > and nowhere else. The RM automatically relaxes locality to rack and * and > assigns non-specified machines to the app. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira