Re: Is it possible to set how many map slots to use on each job submission?

Thanks for the tip Harsh, Arun. What I was exactly looking for!! Regards, Ed 2012/1/18 Arun C Murthy a...@hortonworks.com The CapacityScheduler provides exactly this. Setup 2 queues with appropriate capacities for each: http://hadoop.apache.org/common/docs/r1.0.0/capacity_scheduler.html

Re: Is it possible to set how many map slots to use on each job submission?

The CapacityScheduler provides exactly this. Setup 2 queues with appropriate capacities for each: http://hadoop.apache.org/common/docs/r1.0.0/capacity_scheduler.html Arun On Jan 17, 2012, at 10:57 PM, edward choi wrote: Hi, I often run into situations like this: I am running a very heavy

Is it possible to set how many map slots to use on each job submission?

Hi, I often run into situations like this: I am running a very heavy job(let's say job 1) on a hadoop cluster(which takes many hours). Then something comes up that needs to be done very quickly(let's say job 2). Job 2 only takes a couple of hours when executed on hadoop. But it will take a couple

Re: Is it possible to set how many map slots to use on each job submission?

Edward, You need to invest in configuring a non-FIFO scheduler. FairScheduler may be what you are looking for. Take a look at http://hadoop.apache.org/common/docs/current/fair_scheduler.html for the docs. On 18-Jan-2012, at 12:27 PM, edward choi wrote: Hi, I often run into situations like