Hi If you are using CapacityScheduler, can you try using DominantResourceCalculator i.e configuring below property value in capacity-scheduler.xml file.
<property> <name>yarn.scheduler.capacity.resource-calculator</name> <value>org.apache.hadoop.yarn.util.resource. DominantResourceCalculator </value> </property> The basic Idea it works is as follows ‘if user A runs CPU-heavy tasks and user B runs memory-heavy tasks, it attempts to equalize CPU share of user A with Memory-share of user B’ See Java Doc https://apache.googlesource.com/hadoop-common/+/60e3b885ba8344d9f448202f5f2c290b5606ff8f/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/resource/DominantResourceCalculator.java I think this may help you!!! Thanks & Regards Rohith Sharma K S From: twinkle sachdeva [mailto:twinkle.sachd...@gmail.com] Sent: 26 February 2015 14:05 To: USers Hadoop Subject: Node manager contributing to one queue's resources Hi, I have to run two kind of applications, one requiring less cores but more memory ( Application_High_Mem) and another application which requires more cores but less memory ( Application_High_Core). I can use specific queues to submit them to, but that can lead to one node contributing to only one one such application and having some part of resources as idle. Is there a way, let's say extending concept of queues at node manager level to do this or some other way, in which i can achieve it in YARN? Thanks, Twinkle