Thanks everyone. I've opened a VOTE thread.. @Carlo Interesting idea to support Kubernates API. Definitely makes sense. My understanding is that we can emulate specification for a k8 POD within our scheme of things by assigning the same source tag to a set of scheduling requests. Ill dig in to this.
Cheers -Arun On Fri, Jan 26, 2018 at 1:34 AM, Carlo Aldo Curino <carlo.cur...@gmail.com> wrote: > Also I think this can help us close the gap (and surpass) Kubernetes for > complex services (at least for resource management)... It would be awesome > to have a compatibility layer so folks can run Kubernetes natives apps on a > yarn cluster. > > > > > On Jan 26, 2018 1:32 AM, "Carlo Aldo Curino" <carlo.cur...@gmail.com> > wrote: > > +1. I didn't runs tests, but I like the design, and speaking with ops > teams that operate large clusters I hear this is a feature they think is > going to help a lot, so I am very supportive of this effort. > > On Jan 25, 2018 7:08 PM, "Konstantinos Karanasos" <kkarana...@gmail.com> > wrote: > >> Thanks for starting the thread Arun, +1 from me too. >> >> Konstantinos >> >> On Thu, Jan 25, 2018 at 18:54 Weiwei Yang <cheersy...@hotmail.com> wrote: >> >>> +1, thanks for getting to this milestone Arun. >>> I’ve done some basic validations on a 4 nodes cluster, with some general >>> affinity/anti-affinty/cardinality constraints, it worked. I’ve also >>> reviewed the doc, it’s in good shape and very illustrative. >>> >>> Thanks. >>> >>> -- >>> Weiwei >>> >>> On 26 Jan 2018, 10:44 AM +0800, Sunil G <sun...@apache.org>, wrote: >>> +1. >>> >>> Thanks Arun. >>> >>> I did manual testing for check affinity and anti-affinity features with >>> placement allocator. Also checked SLS to see any performance regression, >>> and there are not much difference as Arun mentioned. >>> >>> Thanks all the folks for working on this. Kudos! >>> >>> - Sunil >>> >>> >>> On Fri, Jan 26, 2018 at 5:16 AM Arun Suresh <asur...@apache.org<mailto: >>> asur...@apache.org>> wrote: >>> Hello yarn-dev@ >>> >>> We feel that the YARN-6592 dev branch mostly in shape to be merged into >>> trunk. This branch adds support for placing containers in YARN using rich >>> placement constraints. For example, this can be used by applications to >>> co-locate containers on a node or rack (affinity constraint), spread >>> containers across nodes or racks (anti-affinity constraint), or even >>> specify the maximum number of containers on a node/rack (cardinality >>> constraint). >>> >>> We have integrated this feature into the Distributed-Shell application >>> for feature testing. We have performed end-to-end testing on >>> moderately-sized clusters to verify that constraints work fine. Performance >>> tests have been done via both SLS tests and Capacity Scheduler performance >>> unit tests, and no regression was found. We have opened a JIRA to track >>> Jenkins acceptance of the aggregated patch [2]. Documentation is in the >>> process of being completed [3]. You can also check our design document for >>> more details [4]. >>> >>> Config flags are needed to enable this feature and it should not have >>> any effect on YARN when turned off. Once merged, we plan to work on further >>> improvements, which can be tracked in the umbrella YARN-7812 [5]. >>> >>> Kindly do take a look at the branch and raise issues/concerns that need >>> to be addressed before the merge. >>> >>> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for >>> their contributions to this effort, as well as Subru, Chris, Carlo, and >>> Vinod for their inputs and discussions. >>> >>> Cheers >>> -Arun >>> >>> >>> [1] https://issues.apache.org/jira/browse/YARN-6592 >>> [2] https://issues.apache.org/jira/browse/YARN-7792 >>> [3] https://issues.apache.org/jira/browse/YARN-7780 >>> [4] https://issues.apache.org/jira/secure/attachment/12867869/YA >>> RN-6592-Rich-Placement-Constraints-Design-V1.pdf >>> [5] https://issues.apache.org/jira/browse/YARN-7812 >>> >>> -- >> Konstantinos >> > >
