In that case, the AMRMClient code looks correct to me. Can you share the code you've written against it that's not receiving the correct containers?
On Wed, Nov 13, 2013 at 5:30 PM, gaurav <gau...@datatorrent.com> wrote: > Hi, > > I looked in the trunk and line numbers are 361 and 366. > > Thanks > -Gaurav > > On 11/13/2013 5:04 PM, gaurav wrote: > > I have hadoop-2.2.0 > > Thanks > -Gaurav > > On 11/13/2013 4:59 PM, Sandy Ryza wrote: > > What version are you using? Setting the relax locality to true for nodes > is always correct. For racks, this is not necessarily the case. When I > look at trunk, it is set to true always on line 361 (which is correct), but > on on line 374. > > -Sandy > > > On Wed, Nov 13, 2013 at 4:47 PM, gaurav <gau...@datatorrent.com> wrote: > >> Hi Sandy, >> >> No it is not working for me. As mentioned earlier, AMRMClient is not >> respecting the locality for node and rack in *Line 361 and 374 *and is >> set to true always. >> I am requesting for one container with specified node and rack and >> relaxed locality false. >> >> Thanks >> -Gaurav >> >> On 11/13/2013 4:24 PM, Sandy Ryza wrote: >> >> [moving to user list] >> >> Right. relaxLocality needs to be set on the next level up. It >> determines whether locality can be relaxed to that level. Confusing, I >> know. If you are using AMRMClient, you should be able to accomplish what >> you're looking for by creating an AMRMClient.ContainerRequest that >> specifies a node and sets relax locality to false. Is that not working for >> you? >> >> -Sandy >> >> >> On Wed, Nov 13, 2013 at 4:19 PM, gaurav <gau...@datatorrent.com> wrote: >> >>> Hi Sandy, >>> >>> If I set relaxlocality to true with host name, I don't get the container >>> on specified host even though node has the resources. >>> I am using AMRMClient, only thing is I made following changes to get the >>> containers on the specified node. >>> / >>> //Line: 361 // >>> //Original// >>> // addResourceRequest(req.getPriority(), node, req.getCapability(), req, >>> true);// >>> //Modifiled// >>> // addResourceRequest(req.getPriority(), node, req.getCapability(), req, >>> req.getRelaxLocality());// >>> // >>> //Line 374// >>> //Original// >>> // addResourceRequest(req.getPriority(), rack, req.getCapability(), req, >>> true);// >>> //Modifiled// >>> // addResourceRequest(req.getPriority(), rack, req.getCapability(), req, >>> req.getRelaxLocality());// >>> /// >>> >>> >>> Thanks >>> -Gaurav >>> >>> >>> On 11/13/2013 4:02 PM, Sandy Ryza wrote: >>> >>>> Yeah, specifying a host name with relaxLocality is meaningful. >>>> Schedulers >>>> use delay scheduling ( >>>> http://www.cs.berkeley.edu/~matei/talks/2010/eurosys_delaysched.pdf) to >>>> achieve locality when relaxLocality is on. But it is turned off by >>>> default. The individual scheduler docs have specifics on how to >>>> configure >>>> it. >>>> >>>> Guarav, >>>> Using ResourceRequests directly is not straightforward and error prone. >>>> Is >>>> there a reason that AMRMClient is unsuitable for your needs? >>>> >>>> -Sandy >>>> >>>> >>>> >>>> On Wed, Nov 13, 2013 at 3:55 PM, Thomas Weise <thomas.we...@gmail.com >>>> >wrote: >>>> >>>> Is it possible to specify a particular node and have RM fallback to an >>>>> different node only after making an attempt to allocate for the >>>>> requested >>>>> node? In other words, is the combination of specific host name and >>>>> relaxLocality=TRUE meaningful at all? >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> On Wed, Nov 13, 2013 at 3:23 PM, Alejandro Abdelnur <t...@cloudera.com >>>>> >>>>>> wrote: >>>>>> Gaurav, >>>>>> >>>>>> Setting relaxLocality to FALSE should do it. >>>>>> >>>>>> thanks. >>>>>> >>>>>> >>>>>> On Wed, Nov 13, 2013 at 2:58 PM, gaurav <gau...@datatorrent.com> >>>>>> wrote: >>>>>> >>>>>> Hi, >>>>>>> I am trying to allocate containers on a particular node in Yarn but >>>>>>> >>>>>> Yarn >>>>> >>>>>> is returning me containers on different node although the requested >>>>>>> >>>>>> node >>>>> >>>>>> has resources available. >>>>>>> >>>>>>> I checked into the allocate(AllocateRequest request) function of >>>>>>> ApplicationMasterService and my request is as follows >>>>>>> >>>>>>> *request: ask { priority { priority: 1 } resource_name: "h2" >>>>>>> >>>>>> capability { >>>>> >>>>>> memory: 1000 } num_containers: 2 } ask { priority { priority: 1 } >>>>>>> resource_name: "/default-rack" capability { memory: 1000 } >>>>>>> >>>>>> num_containers: >>>>>> >>>>>>> 2 } ask { priority { priority: 1 } resource_name: "*" capability { >>>>>>> >>>>>> memory: >>>>>> >>>>>>> 1000 } num_containers: 2 } response_id: 1 progress: 0.0* >>>>>>> >>>>>>> but the containers that I am getting back is as follows >>>>>>> [Container: [ContainerId: container_1384381084244_0001_01_000002, >>>>>>> >>>>>> NodeId: >>>>> >>>>>> h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>, >>>>>>> Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 }, >>>>>>> >>>>>> ], >>>>> >>>>>> Container: [ContainerId: container_1384381084244_0001_01_000003, >>>>>>> >>>>>> NodeId: >>>>> >>>>>> h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>, >>>>>>> Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 }, >>>>>>> >>>>>> ]] >>>>> >>>>>> I am attaching the test case that I have written along with the mail. >>>>>>> >>>>>> It >>>>> >>>>>> uses classes under org.apache.hadoop.yarn.server.resourcemanager >>>>>>> >>>>>> package. >>>>> >>>>>> Any pointers would be of great help >>>>>>> >>>>>>> Thanks >>>>>>> Gaurav >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> Alejandro >>>>>> >>>>>> >>> >> >> > > >
