Re: YARN - specify hosts in ContainerRequest

[Brian C. Huffman]

I get a container, but not on the node I'm asking for.

Thanks,
Brian

On 04/12/2013 03:01 PM, Sandy Ryza wrote:

What do you mean when you say it doesn't seem to use the code? That you're not getting containers back?

-Sandy

On Fri, Apr 12, 2013 at 7:44 AM, Brian C. Huffman <bhuff...@etinternational.com <mailto:bhuff...@etinternational.com>> wrote:

    Yes, this is exactly the DistributedShell example in the 2.0.3
    source (except for the added specification of containerNode).

    So, it's doing the following:
              ContainerRequest containerAsk =
    setupContainerAskForRM(askCount);
              resourceManager.addContainerRequest(containerAsk);


            // Send the request to RM
            LOG.info("Asking RM for containers" + ", askCount=" +
    askCount);
            AMResponse amResp = sendContainerAskToRM();

    FYI - the sendContainerAskToRM calls the allocate(progress).

    I don't see any NPEs in the resourcemanager's logs.

    Regards,
    Brian


    On 04/11/2013 02:50 PM, Harsh J wrote:

        Hi Brian,

        Your way of creation seems fine. Are you "adding" this to the
        AMRMClient somewhere though, via [1], right before you call
        allocate(progress)? Cause otherwise this object's not used at all.

        P.s. I'm not sure we'll handle that null for racks all that
        well in
        certain places yet; I've noticed errors in my app when I've missed
        passing elements or passed them as nulls, but I didn't rebase
        it onto
        2.0.3 yet so haven't reported anything so far - will do so
        soon. I'd
        check the RM log to see if there's any WARN logged there with
        an NPE.

        [1] -
        
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest(org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest)
        
<http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest%28org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest%29>

        On Thu, Apr 11, 2013 at 11:30 PM, Brian C. Huffman
        <bhuff...@etinternational.com
        <mailto:bhuff...@etinternational.com>> wrote:

            All,

            I'm starting to work with YARN and I'm trying to make some
            changes to the
            distributedshell example.

            Specifically I'm trying to set the host where the
            distributed shell
            container will run.  I added the following code into
            ApplicationMaster.java:
                 final String[] containerNodes = { "node1" };

                 LOG.info("Trying to request node: " + containerNodes[0]);
                 ContainerRequest request = new
            ContainerRequest(capability,
            containerNodes, null,
                     pri, numContainers);

            And while the log output shows this code is being executed
            ("Trying to
            request..."), it doesn't seem to use it.  Is this
            functionality that isn't
            complete yet in the 2.0.3 yarn alpha?  Or am I doing
            something wrong?

            Thanks,
            Brian