Thanks for starting the thread Arun, +1 from me too. Konstantinos
On Thu, Jan 25, 2018 at 18:54 Weiwei Yang <cheersy...@hotmail.com> wrote: > +1, thanks for getting to this milestone Arun. > I’ve done some basic validations on a 4 nodes cluster, with some general > affinity/anti-affinty/cardinality constraints, it worked. I’ve also > reviewed the doc, it’s in good shape and very illustrative. > > Thanks. > > -- > Weiwei > > On 26 Jan 2018, 10:44 AM +0800, Sunil G <sun...@apache.org>, wrote: > +1. > > Thanks Arun. > > I did manual testing for check affinity and anti-affinity features with > placement allocator. Also checked SLS to see any performance regression, > and there are not much difference as Arun mentioned. > > Thanks all the folks for working on this. Kudos! > > - Sunil > > > On Fri, Jan 26, 2018 at 5:16 AM Arun Suresh <asur...@apache.org<mailto: > asur...@apache.org>> wrote: > Hello yarn-dev@ > > We feel that the YARN-6592 dev branch mostly in shape to be merged into > trunk. This branch adds support for placing containers in YARN using rich > placement constraints. For example, this can be used by applications to > co-locate containers on a node or rack (affinity constraint), spread > containers across nodes or racks (anti-affinity constraint), or even > specify the maximum number of containers on a node/rack (cardinality > constraint). > > We have integrated this feature into the Distributed-Shell application for > feature testing. We have performed end-to-end testing on moderately-sized > clusters to verify that constraints work fine. Performance tests have been > done via both SLS tests and Capacity Scheduler performance unit tests, and > no regression was found. We have opened a JIRA to track Jenkins acceptance > of the aggregated patch [2]. Documentation is in the process of being > completed [3]. You can also check our design document for more details [4]. > > Config flags are needed to enable this feature and it should not have any > effect on YARN when turned off. Once merged, we plan to work on further > improvements, which can be tracked in the umbrella YARN-7812 [5]. > > Kindly do take a look at the branch and raise issues/concerns that need to > be addressed before the merge. > > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for > their contributions to this effort, as well as Subru, Chris, Carlo, and > Vinod for their inputs and discussions. > > Cheers > -Arun > > > [1] https://issues.apache.org/jira/browse/YARN-6592 > [2] https://issues.apache.org/jira/browse/YARN-7792 > [3] https://issues.apache.org/jira/browse/YARN-7780 > [4] > https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf > [5] https://issues.apache.org/jira/browse/YARN-7812 > > -- Konstantinos
