[
https://issues.apache.org/jira/browse/TWILL-87?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Terence Yim updated TWILL-87:
-----------------------------
Labels: features github (was: features github newbie)
> Adding Container Placement control (Placement Policy API)
> ----------------------------------------------------------
>
> Key: TWILL-87
> URL: https://issues.apache.org/jira/browse/TWILL-87
> Project: Apache Twill
> Issue Type: New Feature
> Components: api, yarn
> Affects Versions: 0.3.0-incubating
> Environment: Tested on Hadoop Yarn 2.2 and 2.3 running on
> Ubuntu-nodes (4 GB , 8 Cores) cluster.
> Reporter: Gourav Khaneja
> Assignee: Gourav Khaneja
> Labels: features, github
> Fix For: 0.3.0-incubating
>
>
> Yarn AMRMClient provides API to control container placement. We need to
> enhance Twill API so that user can specify container placement policy. Twill
> could use AMRMClient to try allocating containers according to specified
> placement policy.
> 1. API:
> a. Added Placement Policy API in TwillSpecificationBuilder.
> b. For now, the only Placement policy added is DISTRIBUTED, which tries
> to spawn specified runnables on different hosts.
> c. Added a placeholder for 'PlacementHints' through which user can
> specify host names they want the runnables to go to.
>
> 2. Code Changes:
> a. Allocating Containers for one runnable instance at a time, instead of
> grouping them together based on Resource Specs.
> b. Added PlacementPolicy and PlacementPolicyGroup data structures
> /interface for storing placement policies.
> c. Storing node name too (apart from hostname) in TwillRunResource.
> 3. AMRMClient API Usage
> a. Using Blacklist to support DISTRIBUTED placement policy.
> b. Not using addContainerRequest.relaxLocality = false for now.
> 4. Unit Tests
> a. Added PlacementPolicyTestRun Class with test method
> testDistributedPlacementPolicy.
> b. Running MiniYarnCluster with 3 nodes instead of 1.
> Please review changes in PR - https://github.com/apache/incubator-twill/pull/7
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
