and you can find that by looking at your mapred-site.xml . look for this property
mapred-site.xml: <name>mapred.jobtracker.taskScheduler</name> you should be using the default FIFO or <value>org.apache.hadoop.mapred.FairScheduler</value> or <value>org.apache.hadoop.mapred.CapacityTaskScheduler</value> Depends on what scheduler you are using , please read thru the following link to understand how the scheduler works ( like allocation , preemption etc ) . http://hadoop.apache.org/docs/stable/capacity_scheduler.html http://hadoop.apache.org/docs/stable/fair_scheduler.html > > ---------- Forwarded message ---------- > From: Ravi Kiran <maghamraviki...@gmail.com> > Date: Fri, Aug 30, 2013 at 10:30 AM > Subject: Re: Mappers per job per user > To: user@hive.apache.org > > > Hi Brad, > > I believe you have configured a Capacity Scheduler for scheduling the > jobs rather than the default FIFO scheduler. > > Regards > Ravi Magham > > > On Fri, Aug 30, 2013 at 10:07 PM, Brad Ruderman > <bruder...@radiumone.com>wrote: > >> Hi All- >> I was hoping to gather some insight in how the hadoop (and or hive) job >> scheduler distributes mappers per user. I am running into an issue where I >> see that hadoop (and or hive) is evenly distributing mappers per user >> instead of per job. >> >> For example: >> -We have 1000 mapper capacity >> -10 Jobs are running total under User A >> -Each job is using 100 mappers >> -User B starts a job >> -User B's job is allocated 250 mappers >> -User A's jobs decrease to 75 mappers each instead of 100 >> >> What could be causing this allocation to occur by user and job, instead >> of just by job. I reviewed the hive/hadoop documentation and wasn't able to >> find any references to distributing jobs by user. >> >> All of the jobs are being executed within the hive shell, or using the >> hive command. >> >> Thanks, >> Brad >> > > >
