Thanks for the sharing another view point, will look around it. But seems like, we might need something more specific.
On Thu, Feb 26, 2015 at 2:43 PM, Rohith Sharma K S < rohithsharm...@huawei.com> wrote: > Hi > > > > If you are using CapacityScheduler, can you try using > DominantResourceCalculator i.e configuring below property value in > capacity-scheduler.xml file. > > > > <property> > > <name>yarn.scheduler.capacity.resource-calculator</name> > > <value>org.apache.hadoop.yarn.util.resource. > DominantResourceCalculator </value> > > </property> > > > > The basic Idea it works is as follows ‘if user A runs CPU-heavy tasks > and user B runs memory-heavy tasks, it attempts to equalize CPU share of > user A with Memory-share of user B’ > > > > See Java Doc > > > https://apache.googlesource.com/hadoop-common/+/60e3b885ba8344d9f448202f5f2c290b5606ff8f/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/resource/DominantResourceCalculator.java > > > > I think this may help you!!! > > > > Thanks & Regards > > Rohith Sharma K S > > > > *From:* twinkle sachdeva [mailto:twinkle.sachd...@gmail.com] > *Sent:* 26 February 2015 14:05 > *To:* USers Hadoop > *Subject:* Node manager contributing to one queue's resources > > > > Hi, > > > > I have to run two kind of applications, one requiring less cores but more > memory ( Application_High_Mem) and another application which requires more > cores but less memory ( Application_High_Core). > > > > I can use specific queues to submit them to, but that can lead to one node > contributing to only one one such application and having some part of > resources as idle. > > > > Is there a way, let's say extending concept of queues at node manager > level to do this or some other way, in which i can achieve it in YARN? > > > > Thanks, > > Twinkle >
