Why would we want two different ways of specifying the same thing? On Wed, Feb 24, 2016 at 3:29 PM, Amol Kekre <[email protected]> wrote:
> Stream locality is a stream attribute. Affinity is operator attribute. > Technically they may not be connected. Though operator affinity can be > extended to specify a situation that implicity covers stream locality, it > may be better to let user say the two following statements independently. > > - This stream is heavy on I/O and I want it to be ..._local // This > statement is independent of future additions of operators to this stream > - I need these logical operators to not be (or be) on the same node > > Do note that container_local or node_local stream localities only relate to > connected operators. > > Thks > Amol > > > On Wed, Feb 24, 2016 at 1:54 PM, Pramod Immaneni <[email protected]> > wrote: > > > Would it make sense to deprecate stream locality in favor of this? > > > > On Tue, Feb 23, 2016 at 6:05 PM, Isha Arkatkar <[email protected]> > > wrote: > > > > > Hi Pramod, > > > > > > We can have a list or operators or regex to specify operators list > for > > > affinity rule. Though, internal implementation will translate both > these > > > into list of pairs. Since, having pair of operators helps in validation > > > phase. > > > > > > And yes, validation phase will catch conflicting locality or affinity > > > rules and throw validation exception. > > > > > > Thanks for input regarding unifiers. It is probably alright to not > > include > > > it in the affinity settings. Will leave the current implementation as > is. > > > > > > Thanks! > > > Isha > > > > > > > > > On Tue, Feb 23, 2016 at 5:56 PM, Pramod Immaneni < > [email protected] > > > > > > wrote: > > > > > > > Isha, > > > > > > > > If the implementation would support (eventually) affinity between > more > > > than > > > > 2 logical operators, instead of relying on specifying it as multiple > > > rules > > > > why not make provision in the API to specify more than 2 logical > > > operators? > > > > If there is a stream locality rule that conflicts with the affinity > > rule > > > > will the validation catch this? With unifiers the common case of MxN > > > > locality is already chosen based on downstream operator. In other > cases > > > > such as single downstream or cascade case I am not sure it is all the > > > > important to have it. > > > > > > > > Thanks > > > > > > > > On Tue, Feb 23, 2016 at 5:37 PM, Isha Arkatkar <[email protected] > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I have opened a review only pull request for handling affinity > > > rules > > > > in > > > > > Apex: > > > > > https://github.com/apache/incubator-apex-core/pull/234 > > > > > > > > > > Just wanted to confirm with everyone, if the APIs for specifying > > > > affinity > > > > > rules look alright. Also, wanted to get views on how unifiers > should > > be > > > > > handled in case of operator affinity rules. More details follow: > > > > > > > > > > I have added a list of affinity rules as an attribute in Dag > > context. > > > > For > > > > > now, have added AffinityRule to be specified for operator pairs. We > > can > > > > add > > > > > regex support in next iteration. > > > > > > > > > > Here is sample usage: > > > > > > > > > > List<AffinityRule> rules = new ArrayList<>(); > > > > > // To add node locality between 2 not connected operators rand > > and > > > > > console: > > > > > rules.add(new AffinityRule(Type.AFFINITY, new OperatorPair( > > "rand", > > > > > "console"), Locality.NODE_LOCAL, false)); > > > > > > > > > > // To add anti-affinity between all partitions of rand operator > > with > > > > all > > > > > partitions of passThru operator: > > > > > rules.add(new AffinityRule(Type.ANTI_AFFINITY, new OperatorPair( > > > > "rand", > > > > > "passThru"), Locality.NODE_LOCAL, false)); > > > > > > > > > > // To add anti-affinity between partitions of passThru operator, > > > give > > > > > same operator name in pair: > > > > > rules.add(new AffinityRule(Type.ANTI_AFFINITY, new > OperatorPair( > > > > > "passThru", "passThru"), Locality.NODE_LOCAL, false)); > > > > > > > > > > // Set the rules in dag context > > > > > dag.setAttribute(DAGContext.AFFINITY_RULES, rules); > > > > > > > > > > > > > > > Please find a sample application using these affinity rules here: > > > > > > https://github.com/ishark/Apex-Samples/tree/master/affinity-example > > > > > > > > > > > > > > > The actual implementation of affinity rules heavily depends node > > > specific > > > > > requests, that was already being handled in > > StreamingAppMasterService. > > > To > > > > > handle node requests for cloudera, I have added an override to > > > blacklist > > > > > all other nodes except the ones where container request is to be > > > issued. > > > > > > > > > > There is one open question I wanted to bring up: In case of > > partitioned > > > > > operators, should the unifiers also follow affinity or > anti-affinity > > > > rules? > > > > > Or should they be independent. For now, I have kept them > independent > > > and > > > > > only the actual operators follow affinity rules. Please suggest > what > > > > would > > > > > make more sense from user's specification perspective. > > > > > > > > > > Thanks, > > > > > Isha > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 1, 2016 at 2:12 PM, Isha Arkatkar < > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi folks, > > > > > > > > > > > > Summarizing the proposal for affinity/anti-affinity rules in > > apex > > > as > > > > > > per discussions in this mail thread. Please suggest if I missed > > > > > something. > > > > > > > > > > > > * For configuration:* > > > > > > > > > > > > - We will have application level affinity/anti-affinity rules. > > For > > > > the > > > > > > first iteration, we can support for (anti-)affinity among > operators > > > > > within > > > > > > single application. We can revisit it in the next iteration to > > > support > > > > > > across applications. > > > > > > > > > > > > - Each rule will consist of 4 components: > > > > > > <List of operators>, AFFINITY/ANTI-AFFINITY, > STRICT/RELAXED > > > > > policy, > > > > > > CONTAINER/NODE/RACK > > > > > > I have checked that Apex supports container locality between 2 > > > > operators > > > > > > that are not connected by Steam. Though did not find an API for > > > setting > > > > > it > > > > > > through application. So, affinity rules can cover container > > affinity > > > as > > > > > > well. > > > > > > We will not support THREAD locality in app level affinity or > > > > > > anti-affinity, since it does not apply to unconnected operators. > > > > > > > > > > > > The list of operators can also be specified by regex > > > > > > > > > > > > *For Implementation: * > > > > > > > > > > > > - I have verified that Node specific requests are honored by > > > > > HortonWorks > > > > > > for both Fair and Capacity scheduler. So we can go with the > > approach > > > > of > > > > > > node specific container requests to implement affinity rules. > > > > > > > > > > > > - However, we may need to fall back on blacklist approach for > CDH > > > > > distro, > > > > > > as node specific requests may not work well. As: > > > > > > -> With Fair scheduler, node specific requests are not > > > honored > > > > > in > > > > > > CDH 4. Application master keeps sending requests to Yarn, but no > > > > > containers > > > > > > are allocated. > > > > > > -> I could not yet verify the same with Fair and > Capacity > > > > > > Scheduler on CDH 5. Ran into some configuration issues when > setting > > > up > > > > > CDH > > > > > > 5.5.1. Will update the thread with findings. > > > > > > > > > > > > I will try to build a prototype for affinity rules with these > > > things > > > > in > > > > > > mind. Please let me know if anyone would like to collaborate on > > this > > > > > > effort! :) > > > > > > > > > > > > Thanks! > > > > > > Isha > > > > > > > > > > > > On Mon, Jan 25, 2016 at 10:23 AM, Vlad Rozov < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > >> Yes, with ability to refer not only to a host name, but to > > operators > > > > > >> (possibly both in the same and other applications). In general > > this > > > > > should > > > > > >> allow to specify both affinity and anti affinity using > > expressions. > > > > For > > > > > >> example LOCALITY_CONTAINER = "{app1.op1 or app1.op2} and > > > {!app2.op1}". > > > > > >> > > > > > >> > > > > > >> On 1/25/16 09:39, Thomas Weise wrote: > > > > > >> > > > > > >>> Agreed. Originally I thought you wanted both operators on the > > same > > > > > host. > > > > > >>> To > > > > > >>> pin them to the same container the same idea can be applied, > > > through > > > > a > > > > > >>> "LOCALITY_CONTAINER" attribute. > > > > > >>> > > > > > >>> > > > > > >>> On Mon, Jan 25, 2016 at 9:18 AM, Vlad Rozov < > > > [email protected] > > > > > > > > > > >>> wrote: > > > > > >>> > > > > > >>> Works as a workaround and requires two extra ports that > operators > > > > > >>>> designer > > > > > >>>> may not necessarily provide out of the box. We may apply the > > same > > > > hack > > > > > >>>> to > > > > > >>>> anti-affinity and request that any two operators with > > > anti-affinity > > > > > rule > > > > > >>>> are connected by non functional stream, but it does not look > > like > > > a > > > > > good > > > > > >>>> option :). > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> On 1/25/16 08:52, Thomas Weise wrote: > > > > > >>>> > > > > > >>>> Sandeep suggested to connect them with a stream. > > > > > >>>>> > > > > > >>>>> On Mon, Jan 25, 2016 at 8:49 AM, Vlad Rozov < > > > > [email protected] > > > > > > > > > > > >>>>> wrote: > > > > > >>>>> > > > > > >>>>> Sorry, possibly I miss something. X and Z are not directly > > > > connected > > > > > >>>>> by a > > > > > >>>>> > > > > > >>>>>> stream, how can I apply THREAD_LOCAL? > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>>> On 1/25/16 08:44, Thomas Weise wrote: > > > > > >>>>>> > > > > > >>>>>> Why does THREAD_LOCAL not work? > > > > > >>>>>> > > > > > >>>>>>> On Mon, Jan 25, 2016 at 7:43 AM, Vlad Rozov < > > > > > [email protected] > > > > > >>>>>>> > > > > > > >>>>>>> wrote: > > > > > >>>>>>> > > > > > >>>>>>> Neither LOCALITY_HOST or THREAD_LOCAL provides required > > > > > >>>>>>> functionality. > > > > > >>>>>>> The > > > > > >>>>>>> > > > > > >>>>>>> goal is to deploy X and Z to the same container/JVM. X may > be > > > > > >>>>>>>> deployed > > > > > >>>>>>>> on > > > > > >>>>>>>> any node. Z needs to be deployed on the same container as > X, > > > so > > > > X > > > > > >>>>>>>> does > > > > > >>>>>>>> not > > > > > >>>>>>>> have any host affinity, and Z has affinity to X, not to > any > > > > > specific > > > > > >>>>>>>> host. > > > > > >>>>>>>> If database client caches writes, reading inside the same > > JVM > > > > will > > > > > >>>>>>>> be > > > > > >>>>>>>> in > > > > > >>>>>>>> process lookup, while deploying Z to any other host/JVM > > leads > > > to > > > > > >>>>>>>> inter > > > > > >>>>>>>> process or inter host lookup. > > > > > >>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>> On 1/24/16 21:03, Thomas Weise wrote: > > > > > >>>>>>>> > > > > > >>>>>>>> There are the attributes LOCALITY_HOST and LOCALITY_RACK > for > > > an > > > > > >>>>>>>> operator > > > > > >>>>>>>> > > > > > >>>>>>>> to > > > > > >>>>>>>>> achieve precisely what you are describing. The value > would > > be > > > > an > > > > > >>>>>>>>> alias > > > > > >>>>>>>>> that > > > > > >>>>>>>>> can be mapped to physical hosts for allocation. The same > > > could > > > > be > > > > > >>>>>>>>> used > > > > > >>>>>>>>> of > > > > > >>>>>>>>> anti-affinity, by assigning different values. > > > > > >>>>>>>>> > > > > > >>>>>>>>> Stream locality is a special case of expressing affinity > > for > > > 2 > > > > > >>>>>>>>> operators. > > > > > >>>>>>>>> Note that we also need a way to extend this to partitions > > of > > > > the > > > > > >>>>>>>>> same > > > > > >>>>>>>>> operator. > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Sun, Jan 24, 2016 at 8:52 PM, Vlad Rozov < > > > > > >>>>>>>>> [email protected]> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>> > > > > > >>>>>>>>> It may be quite extreme use case, but suppose that > > operator X > > > > > >>>>>>>>> writes > > > > > >>>>>>>>> to > > > > > >>>>>>>>> a > > > > > >>>>>>>>> > > > > > >>>>>>>>> distributed in-memory database and operator Z reads from > > it. > > > Z > > > > is > > > > > >>>>>>>>> not > > > > > >>>>>>>>> > > > > > >>>>>>>>>> directly connected to X. In such cases it may be > necessary > > > to > > > > > let > > > > > >>>>>>>>>> an > > > > > >>>>>>>>>> application request container affinity (deploying to the > > > same > > > > > JVM) > > > > > >>>>>>>>>> for > > > > > >>>>>>>>>> operators X and Z as writes may be cached and reads from > > the > > > > > same > > > > > >>>>>>>>>> JVM > > > > > >>>>>>>>>> will > > > > > >>>>>>>>>> be potentially faster. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Technically the same may be applied to thread local on > > NUMA > > > > > >>>>>>>>>> boxes, as > > > > > >>>>>>>>>> NUMA > > > > > >>>>>>>>>> aware deployment of containers is not supported. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Vlad > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> On 1/23/16 18:01, Yogi Devendra wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> @Isha > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> In my opinion: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> THREAD_LOCAL, CONTAINER_LOCAL on stream is a special > case > > > of > > > > > >>>>>>>>>>> generic > > > > > >>>>>>>>>>> rules > > > > > >>>>>>>>>>> for Operator X and Operator Y. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> We can say that, THREAD_LOCAL, CONTAINER_LOCAL would be > > > > > >>>>>>>>>>> applicable > > > > > >>>>>>>>>>> only > > > > > >>>>>>>>>>> if > > > > > >>>>>>>>>>> operator X and Y are connected by stream. But, way to > > > express > > > > > >>>>>>>>>>> this > > > > > >>>>>>>>>>> should > > > > > >>>>>>>>>>> be similar to other rules for affinity. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> ~ Yogi > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On 24 January 2016 at 03:49, Isha Arkatkar < > > > > > [email protected] > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Hey Chinmay, > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> I certainly agree on common set of rules for > > > > > configuring > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> affinity! > > > > > >>>>>>>>>>>> Well > > > > > >>>>>>>>>>>> put by a concrete example. :) > > > > > >>>>>>>>>>>> Only thing I would like to point is: affinity > of > > > > > >>>>>>>>>>>> operators > > > > > >>>>>>>>>>>> should > > > > > >>>>>>>>>>>> not > > > > > >>>>>>>>>>>> cover thread and container locality. Since this is > only > > > > > >>>>>>>>>>>> applicable > > > > > >>>>>>>>>>>> when > > > > > >>>>>>>>>>>> operators are connected by stream. So, it makes sense > to > > > > have > > > > > >>>>>>>>>>>> it on > > > > > >>>>>>>>>>>> Stream > > > > > >>>>>>>>>>>> rather than in common configuration. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> And yes, DAG.validate should only check for > > > REQUIRED > > > > or > > > > > >>>>>>>>>>>> STRICT > > > > > >>>>>>>>>>>> policy. We > > > > > >>>>>>>>>>>> can agree on one of the terminologies STRICT/RELAXED > or > > > > > >>>>>>>>>>>> REQUIRED/PREFERRED. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Thanks! > > > > > >>>>>>>>>>>> Isha > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> On Fri, Jan 22, 2016 at 8:38 PM, Chinmay Kolhatkar < > > > > > >>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Hi Isha, Bhupesh, > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> When I suggested singe affinity rule, I was mainly > > talking > > > > > about > > > > > >>>>>>>>>>>> "how > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> to" > > > > > >>>>>>>>>>>>> of configuration and not of implementation. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> I see locality is in a way suggesting an affinity of > > > > > operators. > > > > > >>>>>>>>>>>>> They're > > > > > >>>>>>>>>>>>> close terminologies. > > > > > >>>>>>>>>>>>> By configuring a locality on stream, we're also, in a > > > way, > > > > > >>>>>>>>>>>>> defining > > > > > >>>>>>>>>>>>> affinity of operators. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Until now, only locality was there and hence was > > straight > > > > > >>>>>>>>>>>>> forward > > > > > >>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>> configuration for user. > > > > > >>>>>>>>>>>>> Tomorrow, when anti-affinity configuration comes up, > > one > > > > > might > > > > > >>>>>>>>>>>>> get > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> confused > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> on how to best use both locality & anti-affinity. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> Hence suggested to make both (locality/affinity & > > > > > >>>>>>>>>>>> anti-affinity) as > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> part > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> of single configuration. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> Suggestion is to have a more commonly adopted > > > configuration > > > > > >>>>>>>>>>>> which > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> admins > > > > > >>>>>>>>>>>>> and developer's are familiar with. > > > > > >>>>>>>>>>>>> Again referring to vShere Hypervisor's affinity > rules. > > I > > > > > think > > > > > >>>>>>>>>>>>> they > > > > > >>>>>>>>>>>>> have > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> single configuration which does both. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> Having said that, here is a quick suggestion on how > both > > > can > > > > > be > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> achieved > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> a single configuration: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> CATEGORY TYPE POLICY > ENTITIES > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Affinity THREAD REQUIRED O1, > O2 > > > > > >>>>>>>>>>>>> //Meaning > > > > > >>>>>>>>>>>>> Operator1 & Operator2 should be thread local > > > > > >>>>>>>>>>>>> Affinity NODE PREFERRED O3, > O4 > > > > > >>>>>>>>>>>>> //Meaning > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> O3 & > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> O4 are preferred to be in node local > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> AntiAffinity NODE REQUIRED O1, > O4 > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> //Meaning > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> O1 & > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> O4 should not be on same node. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> AntiAffinity RACK PREFERRED O2, O4 > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> //Meaning > > > > > >>>>>>>>>>>>> O2 > > > > > >>>>>>>>>>>>> & > > > > > >>>>>>>>>>>>> O4 are preferred not to be on same rack. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Linux setting affinity of CPUs for threads is another > > way > > > > of > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> configuration > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> we can take a look at. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> Learning from these commonly adopted configuration > > > pattern, > > > > we > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> should > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> come > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> up with best configuration suitable for distributed > > > > > >>>>>>>>>>>>> environment. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> Idea here is to not have our own configuration and > give > > > > > >>>>>>>>>>>> something > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>> the users. Otherwise such an important concept might > > > > quickly > > > > > >>>>>>>>>>>>> get > > > > > >>>>>>>>>>>>> lost. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Regarding the DAG.validate I think we would need to > add > > > > some > > > > > >>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>> stuff > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>> take care of anti-affinity. > > > > > >>>>>>>>>>>>> Plus, anti-affinity/affinity should be validated at > > > > > >>>>>>>>>>>>> DAG.validate > > > > > >>>>>>>>>>>>> only > > > > > >>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>> the ones which are required. > > > > > >>>>>>>>>>>>> For preferred policies, validation in logical plan > > might > > > > be a > > > > > >>>>>>>>>>>>> early > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> check. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> Chinmay. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Sat, Jan 23, 2016 at 3:15 AM, Isha Arkatkar < > > > > > >>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Hi, > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Thanks for inputs! I like the idea of having > > > single > > > > > set > > > > > >>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> (anti?)affinity rules at dag context level. > > > > > >>>>>>>>>>>>>> There could still be conflicts based on > > Locality > > > > for > > > > > >>>>>>>>>>>>>> Streams > > > > > >>>>>>>>>>>>>> or > > > > > >>>>>>>>>>>>>> Node > > > > > >>>>>>>>>>>>>> locality attribute set for operators. But as Sandeep > > > > > >>>>>>>>>>>>>> suggested, I > > > > > >>>>>>>>>>>>>> also > > > > > >>>>>>>>>>>>>> think Dag.validate should fail in case of > > contradicting > > > > > >>>>>>>>>>>>>> constraints. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> We currently do not have 'affinity' among non > > > stream > > > > > >>>>>>>>>>>>>> operators, as > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Bhupesh > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> pointed out. It is somewhat achievable by requesting > > > node > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> locality > > > > > >>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> operators (Assuming node requests work as expected). > > But > > > > > >>>>>>>>>>>>> should we > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> consider > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> adding affinity specifications support as well along > > > with > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> anti-affinity? > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Regarding specification of attributes from > > > > > dt-site.xml. > > > > > >>>>>>>>>>>>>> We > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>> go > > > > > >>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Json like string or even xml representation for > complex > > > > > >>>>>>>>>>>>> objects. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> What > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> our current behavior for setting Java object > > properties > > > > > >>>>>>>>>>>>>> through > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> XML? > > > > > >>>>>>>>>>>>> We > > > > > >>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> follow the same for this as well. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> As for precedence or ability to satisfy > > > constraints: > > > > > >>>>>>>>>>>>> Right > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> now in > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> normal > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> scenario, if resources are not available for > > allocating > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> containers, > > > > > >>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> keep > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> sending to request till all are obtained. Likewise, > in > > > > case > > > > > of > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> strict > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> anti-affinity policy, we should keep the application > in > > > > > >>>>>>>>>>>>> ACCEPTED > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> state > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> till > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> the anti-affinity constraint is satisfied. For > relaxed > > > > > >>>>>>>>>>>>>> policy, we > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> decide timeout for relaxing the anti-affinity rule. > > > Please > > > > > note > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> this > > > > > >>>>>>>>>>>>>> applies only when we have non-contradicting rules. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks! > > > > > >>>>>>>>>>>>>> Isha > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 5:30 AM, Bhupesh Chawda < > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> [email protected] > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> I agree on having a single set of rules for the > > affinity > > > as > > > > > >>>>>>>>>>>>> well > > > > > >>>>>>>>>>>>> as > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> anti-affinity of operators / partitions on > containers. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> However, I noted the following points: > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> 1. AFAIK, we do not support affinity > > (locality) > > > > in > > > > > a > > > > > >>>>>>>>>>>>>>> general > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> sense. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> The > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> affinity is only for a stream, not for *any* > > two > > > > > >>>>>>>>>>>>> operators. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> So, > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> should also look at the general case and > see > > > how > > > > it > > > > > >>>>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> supported, > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> if > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> there are valid use cases. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> 2. Coming to anti-affinity, we cannot type > > cast > > > it > > > > > as > > > > > >>>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> type of > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> affinity > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> rule. Saying "two operators must be on the > > same > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> container" is > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> very > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> different from saying "these two operators > > must > > > > not > > > > > >>>>>>>>>>>>>>> be on > > > > > >>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>> same > > > > > >>>>>>>>>>>>>>> container". In this case, the second one > is a > > > > much > > > > > >>>>>>>>>>>>>>> more > > > > > >>>>>>>>>>>>>>> relaxed > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> rule > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> as > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> compared to the first one. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> 3. Once we have this generic set of rules, > > there > > > > must > > > > > >>>>>>>>>>>>>> be a > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> "satisfiability" test run before requesting > > > YARN > > > > > for > > > > > >>>>>>>>>>>>>>> containers. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> If > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> request is not satisfiable, then there is no > > > point > > > > > >>>>>>>>>>>>> asking > > > > > >>>>>>>>>>>>> YARN > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> allocate > > > > > >>>>>>>>>>>>>>> containers in this manner. In case it is > not > > > > > >>>>>>>>>>>>>>> satisfiable, we > > > > > >>>>>>>>>>>>>>> must > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> also > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> have > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> a default order in which the rules can be > > > > "relaxed" > > > > > >>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> request > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> made > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> satisfiable. For example, some very strict > > rules > > > > may > > > > > be > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> ignored, > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> or > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> made > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> less constraining ( for example "on the same > > > > > >>>>>>>>>>>>> container" => > > > > > >>>>>>>>>>>>> "on > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> same > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> node"). > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> -Bhupesh > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 2:54 PM, Aniruddha > Thombare < > > > > > >>>>>>>>>>>>>>> [email protected]> wrote: > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> +1 On Chinmay's suggestion about having single set > of > > > > > >>>>>>>>>>>>>>> affinity > > > > > >>>>>>>>>>>>>>> rules. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Aniruddha > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:57 PM, Sandeep Deshmukh > < > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> [email protected] > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Between 2 operators, if one configures > > thread/container > > > > > local > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> anti-affinity as well, which one will take affect? > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> The DAG validation step should error out in this > > > case. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> +1 on suggestion by Chinmay to name it "Affinity > > > Rules" > > > > > >>>>>>>>>>>>>>> than > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> anti-affinity. We are just extending our container > > > > > allocation > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> scheme > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> support containers not to be allocated together. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Sandeep > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:43 PM, Chinmay > Kolhatkar > > < > > > > > >>>>>>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Hi Isha, > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Couple of points: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> 1. About the interface to configuring > > anti-affinity, > > > as > > > > > per > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> suggestion > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> above there are 2 different way to configure > > > locality > > > > > and > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> anti-affinity: > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> i.e. dag.setAttribute - for anti-affinity & > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> dag.addStream(...).setLocality for > locality. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Between 2 operators, if one configures > > > thread/container > > > > > >>>>>>>>>>>>>>>>>> local > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> anti-affinity as well, which one will take > affect? > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> 2. Consider there could be such confusion as > above, > > > > would > > > > > >>>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> make > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> sense > > > > > >>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> have a single API which takes care of both > > > anti-affinity > > > > > and > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> locality. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> This > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> way, one is configurable. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> 3. This point is coming from how VM affinity is > > > > configured > > > > > >>>>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> vSphere. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> The VMs are configured affinity are called as > > > > "affinity > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> rules" > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> not > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> "anti-affinity rules". Ultimately idea is to > > allocate > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> processing > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> nodes. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Via "VM-VM affinity rules", anti-affinity is also > > > > > configured. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> But > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> there > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> a single set of rule definition for both affinity > > > > (similar > > > > > to > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> locality > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> our case) and anti-affinity. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Would it be a better approach for configuring > > locality > > > > > rules > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> anti-affinity rules in a single rule and call it > > > > > "affinity > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> rule". > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>>> Chinmay. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 12:24 PM, Yogi Devendra < > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> [email protected] > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> @Isha > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> I understand that anti-affinity across > application > > is > > > > not > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> straight-forward. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> It would be OK even if we do not have it in > > > iteration > > > > > 1. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> But, for attributes syntax; I still think that > > Java > > > > > object > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> should > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> avoided as they will be hard to configure from > > > > > >>>>>>>>>>>>>>>>> dt-site.xml or > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> other > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> config > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> files. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Other suggestion for this could be JSON > > representation > > > > of > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> String > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> array: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> ["O2", "O3"]. (If operator names has some > special > > > > > >>>>>>>>>>>>>>>>> characters > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> like > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> " > > > > > >>>>>>>>>>>>>>>>> or [ > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> or , those will be escaped in the JSON > > > representation.) > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Not sure if others agree on this; but attribute > > syntax > > > > > >>>>>>>>>>>>>>>>> should > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> finalized > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> in iteration 1 to avoid backward compatibility > > > issues > > > > > >>>>>>>>>>>>>>>>> later. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> ~ Yogi > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> On 22 January 2016 at 00:43, Thomas Weise < > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Node based requests is the best approach - if it > > > works > > > > > :-) > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Blacklisting will require to allocate the > containers > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> sequentially. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> It > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> will > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> work, but slow down application startup, > especially > > > for > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> larger > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> topologies. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On Thu, Jan 21, 2016 at 10:42 AM, Isha > Arkatkar < > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Hi, > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Should we consider the node based > requests > > if > > > > it > > > > > >>>>>>>>>>>>>>>>>>> works > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Capacity > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Scheduler or avoid 2b approach altogether? I > > > checked > > > > > >>>>>>>>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> node > > > > > >>>>>>>>>>>>>>>>> requests > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> do > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> not work with fair scheduler on CDH cluster. > Yarn > > > does > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> not > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> return > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> any > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> container if hostname is given in the container > > > > request. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> I > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> am > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> trying > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> setup a small virtual hortonworks cluster to check > > the > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> this > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> behavior > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> on > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> that. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> YARN-2027 < > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/YARN-2027 > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> mentioned > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> container requests are not honored in capacity > > > > scheduler > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> too. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> But I > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> am > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> not > > > > > >>>>>>>>>>>>>>>>> sure if it is because of distro dependent issue. > > > Please > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> share > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> insights. > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> @Vlad, Adding support for regular expression > > > sounds > > > > > >>>>>>>>>>>>>>>>>>> good. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> We > > > > > >>>>>>>>>>>>>>>>> could > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> translate to list of operator names internally > > based > > > > on > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> regex. > > > > > >>>>>>>>>>>>>>>>> @Yogi, I went with a list of strings for > attribute > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> because > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> "O2, > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> O3" > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> could > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> be a valid single operator name too :) > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> I am not sure of ways to implement anti-affinity > > > across > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> application. > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Though > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> something to consider for later iteration. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Isha > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:59 PM, Thomas > Weise < > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > https://issues.apache.org/jira/browse/SLIDER-82 > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:56 PM, Thomas Weise < > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> The point was that containers are taken away > > from > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> other > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> apps > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> may > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> have > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> to discard work etc. It's not good style to claim > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> resources > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> not > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> use > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> them eventually :-) > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> For this feature it is necessary to look at the > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> scheduler > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> capabilities/semantics and limitations. For > > > > > example, > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> don't > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> bet > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> exclusively > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> on node requests if the goal is for it to work > > with > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> FairScheduler. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> Also look at Slider, which just recently added > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> support > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> anti-affinity > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> (using node requests). When you run it on the > > CDH > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> cluster, > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> probably > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> won't work... > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 3:19 PM, Pramod > Immaneni < > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> [email protected] > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> Once released won't the containers be > > available > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> again > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> pool. > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> This > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> would only be optional and not mandatory. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 2:02 PM, Thomas Weise > < > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> [email protected] > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> How about also supporting a minor variation > > of > > > it > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> as > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> an > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> option > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> where it greedily gets the total number of > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> containers > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> discards > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> ones > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> can't use and repeats the process for the > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> remaining > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> till > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> everything > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> has > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> been allocated. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> This is problematic as with resource preemption > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> these > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> containers > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> will > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> potentially taken away from other applications > > and > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> then > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> thrown > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> away. > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> Also does it make sense to support > > anti-cluster > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> affinity? > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 1:21 PM, Isha > > > Arkatkar > > > > < > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Hi all, > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> We want add support for > > Anti-affinity > > > in > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Apex > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> allow > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> applications > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> launch specific physical operators on > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> different > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> nodes(APEXCORE-10 > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> < > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > https://issues.apache.org/jira/browse/APEXCORE-10 > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> ). > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Want > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> request > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> your > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> suggestions/ideas for the same! > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> The reasons for using anti-affinity > in > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> be: > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> ensure > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> reliability, for performance reasons (such as > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> application > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> may > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> not > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> want > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> 2 > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> i/o intensive operators to land on the same > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> node > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> improve > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> performance) > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> or > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> for some application specific constraints(for > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> example, > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> 2 > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> partitions > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> cannot > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> be run on the same node since they use same > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> port > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> number). > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> This > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> general rationale for adding Anti-affinity > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> support. > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> Since, Yarn does not support anti-affinity > > yet > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> (YARN-1042 > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> < > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> > > https://issues.apache.org/jira/browse/YARN-1042 > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> ), > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> need > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> implement > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> the logic in AM. Wanted to get your views on > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> following > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> aspects > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> this > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> implementation: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> *1. How to specify anti-affinity for physical > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators/partitions > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> application:* > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> One way for this is to have an > > > > attribute > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> setting > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> at > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> the logical operator context. And an > operator > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> set > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> this > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> attribute > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> list of operator names which should not be > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> collocated. > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> Consider dag with 3 operators: > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> TestOperator o1 = > > > > > dag.addOperator("O1", > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> TestOperator o2 = > > dag.addOperator("O2", > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> TestOperator o3 = dag.addOperator("O3", > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> To set anti-affinity for O1 operator: > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> dag.setAttribute(o1, > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> OperatorContext.ANTI_AFFINITY, > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> ArrayList<String>(Arrays.asList("O2", > > > "O3"))); > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> This would mean O1 should not be > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> allocated > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> on > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> nodes > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> containing > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> operators O2 and O3. This applies to all > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> allocated > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> partitions > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> O1, > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> O2, > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> O3. > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> Also, if same operator name is part > > of > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> list, > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > > > >> > > > > > > > > > > > > > > > > > > > > >
