These are two diff ways of specifying something similar, but not always same. For example, how do I specify the following
This stream has way to high throughput. Force all operators that connect to it today (or in future) to be container local. With stream locality I get that as operator ports are added to the stream, they automatically inherit stream locality. Thks, Amol On Wed, Feb 24, 2016 at 3:37 PM, Pramod Immaneni <[email protected]> wrote: > To explain the affinity support below seems to be a super-set that can > consume the stream locality setting whether that was the original intention > or not. > > On Wed, Feb 24, 2016 at 3:35 PM, Pramod Immaneni <[email protected]> > wrote: > > > Why would we want two different ways of specifying the same thing? > > > > On Wed, Feb 24, 2016 at 3:29 PM, Amol Kekre <[email protected]> > wrote: > > > >> Stream locality is a stream attribute. Affinity is operator attribute. > >> Technically they may not be connected. Though operator affinity can be > >> extended to specify a situation that implicity covers stream locality, > it > >> may be better to let user say the two following statements > independently. > >> > >> - This stream is heavy on I/O and I want it to be ..._local // This > >> statement is independent of future additions of operators to this stream > >> - I need these logical operators to not be (or be) on the same node > >> > >> Do note that container_local or node_local stream localities only relate > >> to > >> connected operators. > >> > >> Thks > >> Amol > >> > >> > >> On Wed, Feb 24, 2016 at 1:54 PM, Pramod Immaneni < > [email protected]> > >> wrote: > >> > >> > Would it make sense to deprecate stream locality in favor of this? > >> > > >> > On Tue, Feb 23, 2016 at 6:05 PM, Isha Arkatkar <[email protected]> > >> > wrote: > >> > > >> > > Hi Pramod, > >> > > > >> > > We can have a list or operators or regex to specify operators > list > >> for > >> > > affinity rule. Though, internal implementation will translate both > >> these > >> > > into list of pairs. Since, having pair of operators helps in > >> validation > >> > > phase. > >> > > > >> > > And yes, validation phase will catch conflicting locality or > affinity > >> > > rules and throw validation exception. > >> > > > >> > > Thanks for input regarding unifiers. It is probably alright to not > >> > include > >> > > it in the affinity settings. Will leave the current implementation > as > >> is. > >> > > > >> > > Thanks! > >> > > Isha > >> > > > >> > > > >> > > On Tue, Feb 23, 2016 at 5:56 PM, Pramod Immaneni < > >> [email protected] > >> > > > >> > > wrote: > >> > > > >> > > > Isha, > >> > > > > >> > > > If the implementation would support (eventually) affinity between > >> more > >> > > than > >> > > > 2 logical operators, instead of relying on specifying it as > multiple > >> > > rules > >> > > > why not make provision in the API to specify more than 2 logical > >> > > operators? > >> > > > If there is a stream locality rule that conflicts with the > affinity > >> > rule > >> > > > will the validation catch this? With unifiers the common case of > MxN > >> > > > locality is already chosen based on downstream operator. In other > >> cases > >> > > > such as single downstream or cascade case I am not sure it is all > >> the > >> > > > important to have it. > >> > > > > >> > > > Thanks > >> > > > > >> > > > On Tue, Feb 23, 2016 at 5:37 PM, Isha Arkatkar < > >> [email protected]> > >> > > > wrote: > >> > > > > >> > > > > Hi all, > >> > > > > > >> > > > > I have opened a review only pull request for handling > affinity > >> > > rules > >> > > > in > >> > > > > Apex: > >> > > > > https://github.com/apache/incubator-apex-core/pull/234 > >> > > > > > >> > > > > Just wanted to confirm with everyone, if the APIs for > specifying > >> > > > affinity > >> > > > > rules look alright. Also, wanted to get views on how unifiers > >> should > >> > be > >> > > > > handled in case of operator affinity rules. More details follow: > >> > > > > > >> > > > > I have added a list of affinity rules as an attribute in Dag > >> > context. > >> > > > For > >> > > > > now, have added AffinityRule to be specified for operator pairs. > >> We > >> > can > >> > > > add > >> > > > > regex support in next iteration. > >> > > > > > >> > > > > Here is sample usage: > >> > > > > > >> > > > > List<AffinityRule> rules = new ArrayList<>(); > >> > > > > // To add node locality between 2 not connected operators > rand > >> > and > >> > > > > console: > >> > > > > rules.add(new AffinityRule(Type.AFFINITY, new OperatorPair( > >> > "rand", > >> > > > > "console"), Locality.NODE_LOCAL, false)); > >> > > > > > >> > > > > // To add anti-affinity between all partitions of rand > operator > >> > with > >> > > > all > >> > > > > partitions of passThru operator: > >> > > > > rules.add(new AffinityRule(Type.ANTI_AFFINITY, new > >> OperatorPair( > >> > > > "rand", > >> > > > > "passThru"), Locality.NODE_LOCAL, false)); > >> > > > > > >> > > > > // To add anti-affinity between partitions of passThru > >> operator, > >> > > give > >> > > > > same operator name in pair: > >> > > > > rules.add(new AffinityRule(Type.ANTI_AFFINITY, new > >> OperatorPair( > >> > > > > "passThru", "passThru"), Locality.NODE_LOCAL, false)); > >> > > > > > >> > > > > // Set the rules in dag context > >> > > > > dag.setAttribute(DAGContext.AFFINITY_RULES, rules); > >> > > > > > >> > > > > > >> > > > > Please find a sample application using these affinity rules > here: > >> > > > > > >> https://github.com/ishark/Apex-Samples/tree/master/affinity-example > >> > > > > > >> > > > > > >> > > > > The actual implementation of affinity rules heavily depends node > >> > > specific > >> > > > > requests, that was already being handled in > >> > StreamingAppMasterService. > >> > > To > >> > > > > handle node requests for cloudera, I have added an override to > >> > > blacklist > >> > > > > all other nodes except the ones where container request is to be > >> > > issued. > >> > > > > > >> > > > > There is one open question I wanted to bring up: In case of > >> > partitioned > >> > > > > operators, should the unifiers also follow affinity or > >> anti-affinity > >> > > > rules? > >> > > > > Or should they be independent. For now, I have kept them > >> independent > >> > > and > >> > > > > only the actual operators follow affinity rules. Please suggest > >> what > >> > > > would > >> > > > > make more sense from user's specification perspective. > >> > > > > > >> > > > > Thanks, > >> > > > > Isha > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Mon, Feb 1, 2016 at 2:12 PM, Isha Arkatkar < > >> [email protected]> > >> > > > > wrote: > >> > > > > > >> > > > > > Hi folks, > >> > > > > > > >> > > > > > Summarizing the proposal for affinity/anti-affinity rules > in > >> > apex > >> > > as > >> > > > > > per discussions in this mail thread. Please suggest if I > missed > >> > > > > something. > >> > > > > > > >> > > > > > * For configuration:* > >> > > > > > > >> > > > > > - We will have application level affinity/anti-affinity > rules. > >> > For > >> > > > the > >> > > > > > first iteration, we can support for (anti-)affinity among > >> operators > >> > > > > within > >> > > > > > single application. We can revisit it in the next iteration to > >> > > support > >> > > > > > across applications. > >> > > > > > > >> > > > > > - Each rule will consist of 4 components: > >> > > > > > <List of operators>, AFFINITY/ANTI-AFFINITY, > >> STRICT/RELAXED > >> > > > > policy, > >> > > > > > CONTAINER/NODE/RACK > >> > > > > > I have checked that Apex supports container locality between 2 > >> > > > operators > >> > > > > > that are not connected by Steam. Though did not find an API > for > >> > > setting > >> > > > > it > >> > > > > > through application. So, affinity rules can cover container > >> > affinity > >> > > as > >> > > > > > well. > >> > > > > > We will not support THREAD locality in app level affinity or > >> > > > > > anti-affinity, since it does not apply to unconnected > operators. > >> > > > > > > >> > > > > > The list of operators can also be specified by regex > >> > > > > > > >> > > > > > *For Implementation: * > >> > > > > > > >> > > > > > - I have verified that Node specific requests are honored by > >> > > > > HortonWorks > >> > > > > > for both Fair and Capacity scheduler. So we can go with the > >> > approach > >> > > > of > >> > > > > > node specific container requests to implement affinity rules. > >> > > > > > > >> > > > > > - However, we may need to fall back on blacklist approach for > >> CDH > >> > > > > distro, > >> > > > > > as node specific requests may not work well. As: > >> > > > > > -> With Fair scheduler, node specific requests are > not > >> > > honored > >> > > > > in > >> > > > > > CDH 4. Application master keeps sending requests to Yarn, but > no > >> > > > > containers > >> > > > > > are allocated. > >> > > > > > -> I could not yet verify the same with Fair and > >> Capacity > >> > > > > > Scheduler on CDH 5. Ran into some configuration issues when > >> setting > >> > > up > >> > > > > CDH > >> > > > > > 5.5.1. Will update the thread with findings. > >> > > > > > > >> > > > > > I will try to build a prototype for affinity rules with > these > >> > > things > >> > > > in > >> > > > > > mind. Please let me know if anyone would like to collaborate > on > >> > this > >> > > > > > effort! :) > >> > > > > > > >> > > > > > Thanks! > >> > > > > > Isha > >> > > > > > > >> > > > > > On Mon, Jan 25, 2016 at 10:23 AM, Vlad Rozov < > >> > > [email protected]> > >> > > > > > wrote: > >> > > > > > > >> > > > > >> Yes, with ability to refer not only to a host name, but to > >> > operators > >> > > > > >> (possibly both in the same and other applications). In > general > >> > this > >> > > > > should > >> > > > > >> allow to specify both affinity and anti affinity using > >> > expressions. > >> > > > For > >> > > > > >> example LOCALITY_CONTAINER = "{app1.op1 or app1.op2} and > >> > > {!app2.op1}". > >> > > > > >> > >> > > > > >> > >> > > > > >> On 1/25/16 09:39, Thomas Weise wrote: > >> > > > > >> > >> > > > > >>> Agreed. Originally I thought you wanted both operators on > the > >> > same > >> > > > > host. > >> > > > > >>> To > >> > > > > >>> pin them to the same container the same idea can be applied, > >> > > through > >> > > > a > >> > > > > >>> "LOCALITY_CONTAINER" attribute. > >> > > > > >>> > >> > > > > >>> > >> > > > > >>> On Mon, Jan 25, 2016 at 9:18 AM, Vlad Rozov < > >> > > [email protected] > >> > > > > > >> > > > > >>> wrote: > >> > > > > >>> > >> > > > > >>> Works as a workaround and requires two extra ports that > >> operators > >> > > > > >>>> designer > >> > > > > >>>> may not necessarily provide out of the box. We may apply > the > >> > same > >> > > > hack > >> > > > > >>>> to > >> > > > > >>>> anti-affinity and request that any two operators with > >> > > anti-affinity > >> > > > > rule > >> > > > > >>>> are connected by non functional stream, but it does not > look > >> > like > >> > > a > >> > > > > good > >> > > > > >>>> option :). > >> > > > > >>>> > >> > > > > >>>> > >> > > > > >>>> > >> > > > > >>>> On 1/25/16 08:52, Thomas Weise wrote: > >> > > > > >>>> > >> > > > > >>>> Sandeep suggested to connect them with a stream. > >> > > > > >>>>> > >> > > > > >>>>> On Mon, Jan 25, 2016 at 8:49 AM, Vlad Rozov < > >> > > > [email protected] > >> > > > > > > >> > > > > >>>>> wrote: > >> > > > > >>>>> > >> > > > > >>>>> Sorry, possibly I miss something. X and Z are not directly > >> > > > connected > >> > > > > >>>>> by a > >> > > > > >>>>> > >> > > > > >>>>>> stream, how can I apply THREAD_LOCAL? > >> > > > > >>>>>> > >> > > > > >>>>>> > >> > > > > >>>>>> On 1/25/16 08:44, Thomas Weise wrote: > >> > > > > >>>>>> > >> > > > > >>>>>> Why does THREAD_LOCAL not work? > >> > > > > >>>>>> > >> > > > > >>>>>>> On Mon, Jan 25, 2016 at 7:43 AM, Vlad Rozov < > >> > > > > [email protected] > >> > > > > >>>>>>> > > >> > > > > >>>>>>> wrote: > >> > > > > >>>>>>> > >> > > > > >>>>>>> Neither LOCALITY_HOST or THREAD_LOCAL provides required > >> > > > > >>>>>>> functionality. > >> > > > > >>>>>>> The > >> > > > > >>>>>>> > >> > > > > >>>>>>> goal is to deploy X and Z to the same container/JVM. X > >> may be > >> > > > > >>>>>>>> deployed > >> > > > > >>>>>>>> on > >> > > > > >>>>>>>> any node. Z needs to be deployed on the same container > >> as X, > >> > > so > >> > > > X > >> > > > > >>>>>>>> does > >> > > > > >>>>>>>> not > >> > > > > >>>>>>>> have any host affinity, and Z has affinity to X, not to > >> any > >> > > > > specific > >> > > > > >>>>>>>> host. > >> > > > > >>>>>>>> If database client caches writes, reading inside the > same > >> > JVM > >> > > > will > >> > > > > >>>>>>>> be > >> > > > > >>>>>>>> in > >> > > > > >>>>>>>> process lookup, while deploying Z to any other host/JVM > >> > leads > >> > > to > >> > > > > >>>>>>>> inter > >> > > > > >>>>>>>> process or inter host lookup. > >> > > > > >>>>>>>> > >> > > > > >>>>>>>> > >> > > > > >>>>>>>> On 1/24/16 21:03, Thomas Weise wrote: > >> > > > > >>>>>>>> > >> > > > > >>>>>>>> There are the attributes LOCALITY_HOST and > LOCALITY_RACK > >> for > >> > > an > >> > > > > >>>>>>>> operator > >> > > > > >>>>>>>> > >> > > > > >>>>>>>> to > >> > > > > >>>>>>>>> achieve precisely what you are describing. The value > >> would > >> > be > >> > > > an > >> > > > > >>>>>>>>> alias > >> > > > > >>>>>>>>> that > >> > > > > >>>>>>>>> can be mapped to physical hosts for allocation. The > same > >> > > could > >> > > > be > >> > > > > >>>>>>>>> used > >> > > > > >>>>>>>>> of > >> > > > > >>>>>>>>> anti-affinity, by assigning different values. > >> > > > > >>>>>>>>> > >> > > > > >>>>>>>>> Stream locality is a special case of expressing > affinity > >> > for > >> > > 2 > >> > > > > >>>>>>>>> operators. > >> > > > > >>>>>>>>> Note that we also need a way to extend this to > >> partitions > >> > of > >> > > > the > >> > > > > >>>>>>>>> same > >> > > > > >>>>>>>>> operator. > >> > > > > >>>>>>>>> > >> > > > > >>>>>>>>> > >> > > > > >>>>>>>>> On Sun, Jan 24, 2016 at 8:52 PM, Vlad Rozov < > >> > > > > >>>>>>>>> [email protected]> > >> > > > > >>>>>>>>> wrote: > >> > > > > >>>>>>>>> > >> > > > > >>>>>>>>> It may be quite extreme use case, but suppose that > >> > operator X > >> > > > > >>>>>>>>> writes > >> > > > > >>>>>>>>> to > >> > > > > >>>>>>>>> a > >> > > > > >>>>>>>>> > >> > > > > >>>>>>>>> distributed in-memory database and operator Z reads > from > >> > it. > >> > > Z > >> > > > is > >> > > > > >>>>>>>>> not > >> > > > > >>>>>>>>> > >> > > > > >>>>>>>>>> directly connected to X. In such cases it may be > >> necessary > >> > > to > >> > > > > let > >> > > > > >>>>>>>>>> an > >> > > > > >>>>>>>>>> application request container affinity (deploying to > >> the > >> > > same > >> > > > > JVM) > >> > > > > >>>>>>>>>> for > >> > > > > >>>>>>>>>> operators X and Z as writes may be cached and reads > >> from > >> > the > >> > > > > same > >> > > > > >>>>>>>>>> JVM > >> > > > > >>>>>>>>>> will > >> > > > > >>>>>>>>>> be potentially faster. > >> > > > > >>>>>>>>>> > >> > > > > >>>>>>>>>> Technically the same may be applied to thread local > on > >> > NUMA > >> > > > > >>>>>>>>>> boxes, as > >> > > > > >>>>>>>>>> NUMA > >> > > > > >>>>>>>>>> aware deployment of containers is not supported. > >> > > > > >>>>>>>>>> > >> > > > > >>>>>>>>>> Vlad > >> > > > > >>>>>>>>>> > >> > > > > >>>>>>>>>> On 1/23/16 18:01, Yogi Devendra wrote: > >> > > > > >>>>>>>>>> > >> > > > > >>>>>>>>>> @Isha > >> > > > > >>>>>>>>>> > >> > > > > >>>>>>>>>> In my opinion: > >> > > > > >>>>>>>>>> > >> > > > > >>>>>>>>>>> THREAD_LOCAL, CONTAINER_LOCAL on stream is a special > >> case > >> > > of > >> > > > > >>>>>>>>>>> generic > >> > > > > >>>>>>>>>>> rules > >> > > > > >>>>>>>>>>> for Operator X and Operator Y. > >> > > > > >>>>>>>>>>> > >> > > > > >>>>>>>>>>> We can say that, THREAD_LOCAL, CONTAINER_LOCAL would > >> be > >> > > > > >>>>>>>>>>> applicable > >> > > > > >>>>>>>>>>> only > >> > > > > >>>>>>>>>>> if > >> > > > > >>>>>>>>>>> operator X and Y are connected by stream. But, way > to > >> > > express > >> > > > > >>>>>>>>>>> this > >> > > > > >>>>>>>>>>> should > >> > > > > >>>>>>>>>>> be similar to other rules for affinity. > >> > > > > >>>>>>>>>>> > >> > > > > >>>>>>>>>>> ~ Yogi > >> > > > > >>>>>>>>>>> > >> > > > > >>>>>>>>>>> On 24 January 2016 at 03:49, Isha Arkatkar < > >> > > > > [email protected] > >> > > > > >>>>>>>>>>> > > >> > > > > >>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>> > >> > > > > >>>>>>>>>>> Hey Chinmay, > >> > > > > >>>>>>>>>>> > >> > > > > >>>>>>>>>>> I certainly agree on common set of rules for > >> > > > > configuring > >> > > > > >>>>>>>>>>> > >> > > > > >>>>>>>>>>> affinity! > >> > > > > >>>>>>>>>>>> Well > >> > > > > >>>>>>>>>>>> put by a concrete example. :) > >> > > > > >>>>>>>>>>>> Only thing I would like to point is: > >> affinity of > >> > > > > >>>>>>>>>>>> operators > >> > > > > >>>>>>>>>>>> should > >> > > > > >>>>>>>>>>>> not > >> > > > > >>>>>>>>>>>> cover thread and container locality. Since this is > >> only > >> > > > > >>>>>>>>>>>> applicable > >> > > > > >>>>>>>>>>>> when > >> > > > > >>>>>>>>>>>> operators are connected by stream. So, it makes > >> sense to > >> > > > have > >> > > > > >>>>>>>>>>>> it on > >> > > > > >>>>>>>>>>>> Stream > >> > > > > >>>>>>>>>>>> rather than in common configuration. > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> And yes, DAG.validate should only check for > >> > > REQUIRED > >> > > > or > >> > > > > >>>>>>>>>>>> STRICT > >> > > > > >>>>>>>>>>>> policy. We > >> > > > > >>>>>>>>>>>> can agree on one of the terminologies > STRICT/RELAXED > >> or > >> > > > > >>>>>>>>>>>> REQUIRED/PREFERRED. > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Thanks! > >> > > > > >>>>>>>>>>>> Isha > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> On Fri, Jan 22, 2016 at 8:38 PM, Chinmay Kolhatkar > < > >> > > > > >>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Hi Isha, Bhupesh, > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> When I suggested singe affinity rule, I was mainly > >> > talking > >> > > > > about > >> > > > > >>>>>>>>>>>> "how > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> to" > >> > > > > >>>>>>>>>>>>> of configuration and not of implementation. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> I see locality is in a way suggesting an affinity > of > >> > > > > operators. > >> > > > > >>>>>>>>>>>>> They're > >> > > > > >>>>>>>>>>>>> close terminologies. > >> > > > > >>>>>>>>>>>>> By configuring a locality on stream, we're also, > in > >> a > >> > > way, > >> > > > > >>>>>>>>>>>>> defining > >> > > > > >>>>>>>>>>>>> affinity of operators. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Until now, only locality was there and hence was > >> > straight > >> > > > > >>>>>>>>>>>>> forward > >> > > > > >>>>>>>>>>>>> in > >> > > > > >>>>>>>>>>>>> configuration for user. > >> > > > > >>>>>>>>>>>>> Tomorrow, when anti-affinity configuration comes > up, > >> > one > >> > > > > might > >> > > > > >>>>>>>>>>>>> get > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> confused > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> on how to best use both locality & anti-affinity. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Hence suggested to make both (locality/affinity & > >> > > > > >>>>>>>>>>>> anti-affinity) as > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> a > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> part > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> of single configuration. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Suggestion is to have a more commonly adopted > >> > > configuration > >> > > > > >>>>>>>>>>>> which > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> admins > >> > > > > >>>>>>>>>>>>> and developer's are familiar with. > >> > > > > >>>>>>>>>>>>> Again referring to vShere Hypervisor's affinity > >> rules. > >> > I > >> > > > > think > >> > > > > >>>>>>>>>>>>> they > >> > > > > >>>>>>>>>>>>> have > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> a > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> single configuration which does both. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Having said that, here is a quick suggestion on how > >> both > >> > > can > >> > > > > be > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> achieved > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> in > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> a single configuration: > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> CATEGORY TYPE POLICY > >> ENTITIES > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Affinity THREAD REQUIRED > >> O1, O2 > >> > > > > >>>>>>>>>>>>> //Meaning > >> > > > > >>>>>>>>>>>>> Operator1 & Operator2 should be thread local > >> > > > > >>>>>>>>>>>>> Affinity NODE PREFERRED > >> O3, O4 > >> > > > > >>>>>>>>>>>>> //Meaning > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> O3 & > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> O4 are preferred to be in node local > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> AntiAffinity NODE REQUIRED > O1, > >> O4 > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> //Meaning > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> O1 & > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> O4 should not be on same node. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> AntiAffinity RACK PREFERRED O2, > O4 > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> //Meaning > >> > > > > >>>>>>>>>>>>> O2 > >> > > > > >>>>>>>>>>>>> & > >> > > > > >>>>>>>>>>>>> O4 are preferred not to be on same rack. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Linux setting affinity of CPUs for threads is > >> another > >> > way > >> > > > of > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> configuration > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> we can take a look at. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Learning from these commonly adopted configuration > >> > > pattern, > >> > > > we > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> should > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> come > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> up with best configuration suitable for > distributed > >> > > > > >>>>>>>>>>>>> environment. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Idea here is to not have our own configuration and > >> give > >> > > > > >>>>>>>>>>>> something > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> new > >> > > > > >>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>> the users. Otherwise such an important concept > might > >> > > > quickly > >> > > > > >>>>>>>>>>>>> get > >> > > > > >>>>>>>>>>>>> lost. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Regarding the DAG.validate I think we would need > to > >> add > >> > > > some > >> > > > > >>>>>>>>>>>>> new > >> > > > > >>>>>>>>>>>>> stuff > >> > > > > >>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>> take care of anti-affinity. > >> > > > > >>>>>>>>>>>>> Plus, anti-affinity/affinity should be validated > at > >> > > > > >>>>>>>>>>>>> DAG.validate > >> > > > > >>>>>>>>>>>>> only > >> > > > > >>>>>>>>>>>>> for > >> > > > > >>>>>>>>>>>>> the ones which are required. > >> > > > > >>>>>>>>>>>>> For preferred policies, validation in logical plan > >> > might > >> > > > be a > >> > > > > >>>>>>>>>>>>> early > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> check. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Thanks, > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>> Chinmay. > >> > > > > >>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> On Sat, Jan 23, 2016 at 3:15 AM, Isha Arkatkar < > >> > > > > >>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Hi, > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Thanks for inputs! I like the idea of > having > >> > > single > >> > > > > set > >> > > > > >>>>>>>>>>>>> of > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> (anti?)affinity rules at dag context level. > >> > > > > >>>>>>>>>>>>>> There could still be conflicts based on > >> > Locality > >> > > > for > >> > > > > >>>>>>>>>>>>>> Streams > >> > > > > >>>>>>>>>>>>>> or > >> > > > > >>>>>>>>>>>>>> Node > >> > > > > >>>>>>>>>>>>>> locality attribute set for operators. But as > >> Sandeep > >> > > > > >>>>>>>>>>>>>> suggested, I > >> > > > > >>>>>>>>>>>>>> also > >> > > > > >>>>>>>>>>>>>> think Dag.validate should fail in case of > >> > contradicting > >> > > > > >>>>>>>>>>>>>> constraints. > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> We currently do not have 'affinity' among > non > >> > > stream > >> > > > > >>>>>>>>>>>>>> operators, as > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> Bhupesh > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> pointed out. It is somewhat achievable by > >> requesting > >> > > node > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> locality > >> > > > > >>>>>>>>>>>>> for > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> operators (Assuming node requests work as > expected). > >> > But > >> > > > > >>>>>>>>>>>>> should we > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> consider > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> adding affinity specifications support as well > >> along > >> > > with > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> anti-affinity? > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> Regarding specification of attributes from > >> > > > > dt-site.xml. > >> > > > > >>>>>>>>>>>>>> We > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> can > >> > > > > >>>>>>>>>>>>> go > >> > > > > >>>>>>>>>>>>> with > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> Json like string or even xml representation for > >> complex > >> > > > > >>>>>>>>>>>>> objects. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> What > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> is > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> our current behavior for setting Java object > >> > properties > >> > > > > >>>>>>>>>>>>>> through > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> XML? > >> > > > > >>>>>>>>>>>>> We > >> > > > > >>>>>>>>>>>>> can > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> follow the same for this as well. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> As for precedence or ability to satisfy > >> > > constraints: > >> > > > > >>>>>>>>>>>>> Right > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> now in > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> normal > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> scenario, if resources are not available for > >> > allocating > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> containers, > >> > > > > >>>>>>>>>>>>> we > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> keep > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> sending to request till all are obtained. > >> Likewise, in > >> > > > case > >> > > > > of > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> strict > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> anti-affinity policy, we should keep the > >> application in > >> > > > > >>>>>>>>>>>>> ACCEPTED > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> state > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> till > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> the anti-affinity constraint is satisfied. For > >> relaxed > >> > > > > >>>>>>>>>>>>>> policy, we > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> can > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> decide timeout for relaxing the anti-affinity > rule. > >> > > Please > >> > > > > note > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> this > >> > > > > >>>>>>>>>>>>>> applies only when we have non-contradicting > rules. > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> Thanks! > >> > > > > >>>>>>>>>>>>>> Isha > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 5:30 AM, Bhupesh Chawda < > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> [email protected] > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> I agree on having a single set of rules for the > >> > affinity > >> > > as > >> > > > > >>>>>>>>>>>>> well > >> > > > > >>>>>>>>>>>>> as > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> anti-affinity of operators / partitions on > >> containers. > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> However, I noted the following points: > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> 1. AFAIK, we do not support affinity > >> > (locality) > >> > > > in > >> > > > > a > >> > > > > >>>>>>>>>>>>>>> general > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> sense. > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> The > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> affinity is only for a stream, not for > >> *any* > >> > two > >> > > > > >>>>>>>>>>>>> operators. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> So, > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> we > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> should also look at the general case and > >> see > >> > > how > >> > > > it > >> > > > > >>>>>>>>>>>>>>> can > >> > > > > >>>>>>>>>>>>>>> be > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> supported, > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> if > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> there are valid use cases. > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> 2. Coming to anti-affinity, we cannot > type > >> > cast > >> > > it > >> > > > > as > >> > > > > >>>>>>>>>>>>>> a > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> type of > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> affinity > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> rule. Saying "two operators must be on > the > >> > same > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> container" is > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> very > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> different from saying "these two > operators > >> > must > >> > > > not > >> > > > > >>>>>>>>>>>>>>> be on > >> > > > > >>>>>>>>>>>>>>> the > >> > > > > >>>>>>>>>>>>>>> same > >> > > > > >>>>>>>>>>>>>>> container". In this case, the second one > >> is a > >> > > > much > >> > > > > >>>>>>>>>>>>>>> more > >> > > > > >>>>>>>>>>>>>>> relaxed > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> rule > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> as > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> compared to the first one. > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> 3. Once we have this generic set of rules, > >> > there > >> > > > must > >> > > > > >>>>>>>>>>>>>> be a > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> "satisfiability" test run before > >> requesting > >> > > YARN > >> > > > > for > >> > > > > >>>>>>>>>>>>>>> containers. > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> If > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> the > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> request is not satisfiable, then there is > >> no > >> > > point > >> > > > > >>>>>>>>>>>>> asking > >> > > > > >>>>>>>>>>>>> YARN > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> allocate > >> > > > > >>>>>>>>>>>>>>> containers in this manner. In case it is > >> not > >> > > > > >>>>>>>>>>>>>>> satisfiable, we > >> > > > > >>>>>>>>>>>>>>> must > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> also > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> have > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> a default order in which the rules can be > >> > > > "relaxed" > >> > > > > >>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>> the > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> request > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> be > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> made > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> satisfiable. For example, some very strict > >> > rules > >> > > > may > >> > > > > be > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> ignored, > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> or > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> made > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> less constraining ( for example "on the > >> same > >> > > > > >>>>>>>>>>>>> container" => > >> > > > > >>>>>>>>>>>>> "on > >> > > > > >>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>> the > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> same > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> node"). > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> Thanks. > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> -Bhupesh > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 2:54 PM, Aniruddha > >> Thombare < > >> > > > > >>>>>>>>>>>>>>> [email protected]> wrote: > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> +1 On Chinmay's suggestion about having single > >> set of > >> > > > > >>>>>>>>>>>>>>> affinity > >> > > > > >>>>>>>>>>>>>>> rules. > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> Thanks, > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> Aniruddha > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:57 PM, Sandeep > >> Deshmukh < > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> [email protected] > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> Between 2 operators, if one configures > >> > thread/container > >> > > > > local > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> anti-affinity as well, which one will take > >> affect? > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> The DAG validation step should error out in > this > >> > > case. > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> +1 on suggestion by Chinmay to name it > "Affinity > >> > > Rules" > >> > > > > >>>>>>>>>>>>>>> than > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> anti-affinity. We are just extending our > container > >> > > > > allocation > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> scheme > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> support containers not to be allocated > together. > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> Regards, > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> Sandeep > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:43 PM, Chinmay > >> Kolhatkar > >> > < > >> > > > > >>>>>>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> Hi Isha, > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> Couple of points: > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> 1. About the interface to configuring > >> > anti-affinity, > >> > > as > >> > > > > per > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> suggestion > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> above there are 2 different way to configure > >> > > locality > >> > > > > and > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> anti-affinity: > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> i.e. dag.setAttribute - for anti-affinity & > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> dag.addStream(...).setLocality for > >> locality. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> Between 2 operators, if one configures > >> > > thread/container > >> > > > > >>>>>>>>>>>>>>>>>> local > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> anti-affinity as well, which one will take > >> affect? > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> 2. Consider there could be such confusion as > >> above, > >> > > > would > >> > > > > >>>>>>>>>>>>>>>> it > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> make > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> sense > >> > > > > >>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> have a single API which takes care of both > >> > > anti-affinity > >> > > > > and > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> locality. > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> This > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> way, one is configurable. > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> 3. This point is coming from how VM affinity is > >> > > > configured > >> > > > > >>>>>>>>>>>>>>>>> in > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> vSphere. > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> The VMs are configured affinity are called as > >> > > > "affinity > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> rules" > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> not > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> "anti-affinity rules". Ultimately idea is to > >> > allocate > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> processing > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> nodes. > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> Via "VM-VM affinity rules", anti-affinity is > also > >> > > > > configured. > >> > > > > >>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>> But > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> there > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> is > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> a single set of rule definition for both > affinity > >> > > > (similar > >> > > > > to > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> locality > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> in > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> our case) and anti-affinity. > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> Would it be a better approach for configuring > >> > locality > >> > > > > rules > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> anti-affinity rules in a single rule and call > >> it > >> > > > > "affinity > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> rule". > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> Thanks, > >> > > > > >>>>>>>>>>>>>>> Chinmay. > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 12:24 PM, Yogi Devendra > < > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> [email protected] > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> @Isha > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> I understand that anti-affinity across > >> application > >> > is > >> > > > not > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> straight-forward. > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> It would be OK even if we do not have it in > >> > > iteration > >> > > > > 1. > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> But, for attributes syntax; I still think > that > >> > Java > >> > > > > object > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> should > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> be > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> avoided as they will be hard to configure > from > >> > > > > >>>>>>>>>>>>>>>>> dt-site.xml or > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> other > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> config > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> files. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> Other suggestion for this could be JSON > >> > representation > >> > > > of > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> String > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> array: > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> ["O2", "O3"]. (If operator names has some > >> special > >> > > > > >>>>>>>>>>>>>>>>> characters > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> like > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> " > >> > > > > >>>>>>>>>>>>>>>>> or [ > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> or , those will be escaped in the JSON > >> > > representation.) > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> Not sure if others agree on this; but attribute > >> > syntax > >> > > > > >>>>>>>>>>>>>>>>> should > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> be > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> finalized > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> in iteration 1 to avoid backward > compatibility > >> > > issues > >> > > > > >>>>>>>>>>>>>>>>> later. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> ~ Yogi > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> On 22 January 2016 at 00:43, Thomas Weise < > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> Node based requests is the best approach - if > >> it > >> > > works > >> > > > > :-) > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> Blacklisting will require to allocate the > >> containers > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> sequentially. > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> It > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> will > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> work, but slow down application startup, > >> especially > >> > > for > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> larger > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> topologies. > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> On Thu, Jan 21, 2016 at 10:42 AM, Isha > >> Arkatkar < > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> Hi, > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> Should we consider the node based > >> requests > >> > if > >> > > > it > >> > > > > >>>>>>>>>>>>>>>>>>> works > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> with > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> Capacity > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> Scheduler or avoid 2b approach altogether? > I > >> > > checked > >> > > > > >>>>>>>>>>>>>>>>>>> that > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> node > >> > > > > >>>>>>>>>>>>>>>>> requests > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> do > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> not work with fair scheduler on CDH cluster. > >> Yarn > >> > > does > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> not > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> return > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> any > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> container if hostname is given in the > container > >> > > > request. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> I > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> am > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> trying > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> setup a small virtual hortonworks cluster to > >> check > >> > the > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> this > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> behavior > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> on > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> that. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> YARN-2027 < > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> > https://issues.apache.org/jira/browse/YARN-2027 > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> mentioned > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> that > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> container requests are not honored in > capacity > >> > > > scheduler > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> too. > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> But I > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> am > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> not > >> > > > > >>>>>>>>>>>>>>>>> sure if it is because of distro dependent > issue. > >> > > Please > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> share > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> insights. > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> @Vlad, Adding support for regular > expression > >> > > sounds > >> > > > > >>>>>>>>>>>>>>>>>>> good. > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> We > >> > > > > >>>>>>>>>>>>>>>>> could > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> translate to list of operator names > internally > >> > based > >> > > > on > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> regex. > >> > > > > >>>>>>>>>>>>>>>>> @Yogi, I went with a list of strings for > >> attribute > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> because > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> "O2, > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> O3" > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> could > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> be a valid single operator name too :) > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> I am not sure of ways to implement > anti-affinity > >> > > across > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> application. > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> Though > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> something to consider for later iteration. > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> Thanks, > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> Isha > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:59 PM, Thomas > >> Weise < > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> https://issues.apache.org/jira/browse/SLIDER-82 > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:56 PM, Thomas > Weise > >> < > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> The point was that containers are taken > away > >> > from > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> other > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> apps > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> that > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> may > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> have > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> to discard work etc. It's not good style to > >> claim > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> resources > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> not > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> use > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> them eventually :-) > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> For this feature it is necessary to look at > >> the > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> scheduler > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> capabilities/semantics and limitations. > >> For > >> > > > > example, > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> don't > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> bet > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> exclusively > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> on node requests if the goal is for it to > work > >> > with > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> FairScheduler. > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> Also look at Slider, which just recently > added > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> support > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> for > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> anti-affinity > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> (using node requests). When you run it on > the > >> > CDH > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> cluster, > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> it > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> probably > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> won't work... > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 3:19 PM, Pramod > >> Immaneni < > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> [email protected] > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> Once released won't the containers be > >> > available > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> again > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> in > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> the > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> pool. > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> This > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> would only be optional and not mandatory. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> Thanks > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 2:02 PM, Thomas > >> Weise < > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> [email protected] > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> How about also supporting a minor > >> variation > >> > of > >> > > it > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> as > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> an > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> option > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> where it greedily gets the total number > of > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> containers > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> and > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> discards > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> ones > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> it > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> can't use and repeats the process for the > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> remaining > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> till > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> everything > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> has > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> been allocated. > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> This is problematic as with resource > preemption > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> these > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> containers > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> will > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> be > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> potentially taken away from other > >> applications > >> > and > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> then > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> thrown > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> away. > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> Also does it make sense to support > >> > anti-cluster > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> affinity? > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> Thanks > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 1:21 PM, Isha > >> > > Arkatkar > >> > > > < > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> [email protected]> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> wrote: > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> Hi all, > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> We want add support for > >> > Anti-affinity > >> > > in > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Apex > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> allow > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> applications > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> launch specific physical operators on > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> different > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> nodes(APEXCORE-10 > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> < > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > https://issues.apache.org/jira/browse/APEXCORE-10 > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> ). > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> Want > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> request > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> your > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> suggestions/ideas for the same! > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> The reasons for using > anti-affinity > >> in > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> be: > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> ensure > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> reliability, for performance reasons (such as > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> application > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> may > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> not > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> want > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> 2 > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> i/o intensive operators to land on the > same > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> node > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> improve > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> performance) > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> or > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> for some application specific constraints(for > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> example, > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> 2 > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> partitions > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> cannot > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> be run on the same node since they use > same > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> port > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> number). > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> This > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> is > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> the > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> general rationale for adding Anti-affinity > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> support. > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Since, Yarn does not support > >> anti-affinity > >> > yet > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> (YARN-1042 > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> < > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > https://issues.apache.org/jira/browse/YARN-1042 > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> ), > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> we > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> need > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> to > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> implement > >> > > > > >>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>> the logic in AM. Wanted to get your views on > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> following > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> aspects > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> for > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> this > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> implementation: > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> *1. How to specify anti-affinity for > physical > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators/partitions > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> in > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> application:* > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> One way for this is to have an > >> > > > attribute > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> for > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> setting > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> at > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> the logical operator context. And an > >> operator > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> can > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> set > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> this > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> attribute > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> with > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> list of operator names which should not be > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> collocated. > >> > > > > >>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> Consider dag with 3 operators: > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> TestOperator o1 = > >> > > > > dag.addOperator("O1", > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > >> > > > > >>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>> TestOperator o2 = > >> > dag.addOperator("O2", > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> TestOperator o3 = > >> dag.addOperator("O3", > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> To set anti-affinity for O1 operator: > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> dag.setAttribute(o1, > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> OperatorContext.ANTI_AFFINITY, > >> > > > > >>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>> new > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> ArrayList<String>(Arrays.asList("O2", > >> > > "O3"))); > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> This would mean O1 should not > >> be > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> allocated > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> on > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> nodes > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> containing > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> operators O2 and O3. This applies to all > >> > > > > >>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>> allocated > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> partitions > >> > > > > >>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>> of > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> O1, > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>> O2, > >> > > > > >>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>> O3. > >> > > > > >>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>> Also, if same operator name is > >> part > >> > of > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> list, > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > >> > > > > >> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > > >
