[ https://issues.apache.org/jira/browse/YARN-5468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928880#comment-15928880 ]
Konstantinos Karanasos commented on YARN-5468: ---------------------------------------------- Hi [~grey], we have a design doc but we are in the process of updating it to reflect the latest discussions -- I will upload it within the next few days. We have had extended discussions with [~leftnoteasy] to make sure we share the same APIs for defining constraints. In brief, we will share the same constraint expressions (affinity and anti-affinity), but in a way that will allow to add more expressive constraints later. The user will be able to define application-wide constraints when submitting the application through the ApplicationSubmissionContext (YARN-6346). Moreover, given that we consider using such constraints for applications with long-running containers (exclusively, at least in the beginning), we are planning to place containers with constraints in a holistic fashion (looking at multiple container requests and constraints at the same time), trading some scheduling latency for better placement decisions (given that these containers will run for hours/days/months, scheduling latency is not as critical). This placement will be performed outside the Capacity/Fair Scheduler to make sure we don't affect the scheduling latency of existing applications. > Scheduling of long-running applications > --------------------------------------- > > Key: YARN-5468 > URL: https://issues.apache.org/jira/browse/YARN-5468 > Project: Hadoop YARN > Issue Type: New Feature > Components: capacityscheduler, fairscheduler > Reporter: Konstantinos Karanasos > Assignee: Konstantinos Karanasos > Attachments: YARN-5468.prototype.patch > > > This JIRA is about the scheduling of applications with long-running tasks. > It will include adding support to the YARN for a richer set of scheduling > constraints (such as affinity, anti-affinity, cardinality and time > constraints), and extending the schedulers to take them into account during > placement of containers to nodes. > We plan to have both an online version that will accommodate such requests as > they arrive, as well as a Long-running Application Planner that will make > more global decisions by considering multiple applications at once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org