[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7612: -- Attachment: YARN-7612-YARN-6592.012.patch Updating patch based on [~kkaranasos]'s suggestions. > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-YARN-6592.007.patch, > YARN-7612-YARN-6592.008.patch, YARN-7612-YARN-6592.009.patch, > YARN-7612-YARN-6592.010.patch, YARN-7612-YARN-6592.011.patch, > YARN-7612-YARN-6592.012.patch, YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7612: -- Attachment: YARN-7612-YARN-6592.011.patch Updating patch (v011): * Addressed some of [~sunilg]'s comments * Fixed some checkstyles and testcase failures. I decided not to move SampleAlgorithm to test. Since I would like to have a default Algorithm (it does simple anti-affinity after all). In anycase - YARN-7613 will have a proper implementaion which will replace this. I will submit the patch once YARN-6596 is committed. > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-YARN-6592.007.patch, > YARN-7612-YARN-6592.008.patch, YARN-7612-YARN-6592.009.patch, > YARN-7612-YARN-6592.010.patch, YARN-7612-YARN-6592.011.patch, > YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7612: -- Attachment: YARN-7612-YARN-6592.010.patch Updating patch (v010). Rebased based on latest YARN-6596 and simplified the code a bit: Realized we don't really need a completionService - so removed it. > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-YARN-6592.007.patch, > YARN-7612-YARN-6592.008.patch, YARN-7612-YARN-6592.009.patch, > YARN-7612-YARN-6592.010.patch, YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7612: -- Attachment: YARN-7612-YARN-6592.009.patch Updating patch (v009) with some minor refactoring. * Removed the tag indexed map from the BatchedRequest - The algorithm is free to index its Input as it see fit. * We now feedback to the algorithm a blacklist. Essentially, if the scheduler had rejected a placed request, and if retry attempts > 1, then when the Algorthim tries to re-place the request, The node on which it had previously placed (and was rejected by the scheduler) is available to the Algorithm as a set of blacklisted nodes. * I have also added a testcasse to show re-placement. The placement algorithm places on a node that has no capacity, and in the next attempt, it will place on a different node using the blacklist. [~cheersyang], this should demonstrate something similar to what you were asking (although using anti-affinity, not cardinality) > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-YARN-6592.007.patch, > YARN-7612-YARN-6592.008.patch, YARN-7612-YARN-6592.009.patch, > YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7612: -- Attachment: YARN-7612-YARN-6592.008.patch Updating patch after rebasing and applying latest version of YARN-7669. > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-YARN-6592.007.patch, > YARN-7612-YARN-6592.008.patch, YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7612: -- Attachment: YARN-7612-YARN-6592.007.patch Updating patch. Attaching only the Processor and related classes. This now depends on YARN-7669 and YARN-7670 > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-YARN-6592.007.patch, > YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7612) Add Placement Processor Framework
[ https://issues.apache.org/jira/browse/YARN-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Karanasos updated YARN-7612: - Summary: Add Placement Processor Framework (was: Add Placement Processor and planner framework) > Add Placement Processor Framework > - > > Key: YARN-7612 > URL: https://issues.apache.org/jira/browse/YARN-7612 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7612-YARN-6592.001.patch, > YARN-7612-YARN-6592.002.patch, YARN-7612-YARN-6592.003.patch, > YARN-7612-YARN-6592.004.patch, YARN-7612-YARN-6592.005.patch, > YARN-7612-YARN-6592.006.patch, YARN-7612-v2.wip.patch, YARN-7612.wip.patch > > > This introduces a Placement Processor and a Planning algorithm framework to > handle placement constraints and scheduling requests from an app and places > them on nodes. > The actual planning algorithm(s) will be handled in a YARN-7613. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
