YARN currently does not provide any guarantees that a container will be placed where you ask for it. This is being addressed in YARN-392 and YARN-398. For now, the Fair Scheduler and Capacity Scheduler will make a best effort to do this through delay scheduling. If you use the Fair Scheduler, you can turn on delay scheduling by setting yarn.scheduler.fair.locality.threshold.node to a float between 0 and 1, which corresponds to the number of node-RM heartbeats to delay allocating a non-local container, as a fraction of the cluster size.
Sandy On Fri, Apr 12, 2013 at 12:08 PM, Brian C. Huffman < bhuff...@etinternational.com> wrote: > I get a container, but not on the node I'm asking for. > > Thanks, > Brian > > > On 04/12/2013 03:01 PM, Sandy Ryza wrote: > > What do you mean when you say it doesn't seem to use the code? That > you're not getting containers back? > > -Sandy > > On Fri, Apr 12, 2013 at 7:44 AM, Brian C. Huffman < > bhuff...@etinternational.com> wrote: > >> Yes, this is exactly the DistributedShell example in the 2.0.3 source >> (except for the added specification of containerNode). >> >> So, it's doing the following: >> ContainerRequest containerAsk = >> setupContainerAskForRM(askCount); >> resourceManager.addContainerRequest(containerAsk); >> >> >> // Send the request to RM >> LOG.info("Asking RM for containers" + ", askCount=" + askCount); >> AMResponse amResp = sendContainerAskToRM(); >> >> FYI - the sendContainerAskToRM calls the allocate(progress). >> >> I don't see any NPEs in the resourcemanager's logs. >> >> Regards, >> Brian >> >> >> On 04/11/2013 02:50 PM, Harsh J wrote: >> >>> Hi Brian, >>> >>> Your way of creation seems fine. Are you "adding" this to the >>> AMRMClient somewhere though, via [1], right before you call >>> allocate(progress)? Cause otherwise this object's not used at all. >>> >>> P.s. I'm not sure we'll handle that null for racks all that well in >>> certain places yet; I've noticed errors in my app when I've missed >>> passing elements or passed them as nulls, but I didn't rebase it onto >>> 2.0.3 yet so haven't reported anything so far - will do so soon. I'd >>> check the RM log to see if there's any WARN logged there with an NPE. >>> >>> [1] - >>> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest(org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest) >>> >>> On Thu, Apr 11, 2013 at 11:30 PM, Brian C. Huffman >>> <bhuff...@etinternational.com> wrote: >>> >>>> All, >>>> >>>> I'm starting to work with YARN and I'm trying to make some changes to >>>> the >>>> distributedshell example. >>>> >>>> Specifically I'm trying to set the host where the distributed shell >>>> container will run. I added the following code into >>>> ApplicationMaster.java: >>>> final String[] containerNodes = { "node1" }; >>>> >>>> LOG.info("Trying to request node: " + containerNodes[0]); >>>> ContainerRequest request = new ContainerRequest(capability, >>>> containerNodes, null, >>>> pri, numContainers); >>>> >>>> And while the log output shows this code is being executed ("Trying to >>>> request..."), it doesn't seem to use it. Is this functionality that >>>> isn't >>>> complete yet in the 2.0.3 yarn alpha? Or am I doing something wrong? >>>> >>>> Thanks, >>>> Brian >>>> >>>> >>> >>> >> >> > >