I bet you are not setting different Resource-Request-priorities on your requests.
It is a current limitation (https://issues.apache.org/jira/browse/YARN-314) of not being able to support resources of different sizes against a single priority. +Vinod On Nov 3, 2014, at 1:23 AM, Smita Deshpande <smita.deshpa...@cumulus-systems.com> wrote: > Hi All, > I have a single YARN application running on a cluster of 1 master > node(Resourcemanager) and 3 slave nodes (Nodemanagers) - total memory= 24GB, > total vcores= 12. I am using Hadoop 2.4.1 and scheduler is Capacity-Scheduler > with DominantResourceCalculator. This application submits 10 container > requests to ResourceManager through the asynchronous AMRM client ( > AMRMClientAsync). The behavior I observed was that the container request > asking for the most resources gets allocated first. For eg. 9 requests are > for 2GB containers and 1 request is for a 4 GB container, then the 4GB > container gets allocated and here's where it runs into a problem - It just > stops there - it doesn't allocate any containers for all the 2 GB requests! > Once a container is allocated my app runs a task on it and once it is > finished , app submits a fresh container request for this task (the tasks are > of a repeating nature). Then again the RM allocates a 4GB container for this > request, completely ignoring all the requests in between. I don't know if it > is an issue with the AMRMClientAsync class- I haven't tried using the normal > AMRMclient yet. Now I have tasks that require 2 GB and tasks that require 4 > GB memory. Because of this problem it is only running the 4 GB task again and > again. > If all the container requests are identical in terms of resources, then this > problem disappears and RM allocates all requested containers. > > Couldn't find any documentation/known bugs relating to this. I've hit a brick > wall here, any help would be greatly appreciated. Thanks in advance > > -SMita
signature.asc
Description: Message signed with OpenPGP using GPGMail
