Reading assignTasks() in 0.20.2 reveals that the number of map tasks assigned is not limited to 1 per heartbeat.
Cheers On Sun, Apr 11, 2010 at 12:30 PM, Todd Lipcon <t...@cloudera.com> wrote: > Hi Abhishek, > > This behavior is improved by MAPREDUCE-706 I believe (not certain that > that's the JIRA, but I know it's fixed in trunk fairscheduler). These > patches are included in CDH3 (currently in beta) > http://archive.cloudera.com/cdh/3/ > > In general, though, map tasks that are so short are not going to be very > efficient - even with fast assignment there is some constant overhead per > task. > > Thanks > -Todd > > On Sun, Apr 11, 2010 at 11:42 AM, abhishek sharma <absha...@usc.edu> > wrote: > > > Hi all, > > > > I have been using the Hadoop Fair Scheduler for some experiments on a > > 100 node cluster with 2 map slots per node (hence, a total of 200 map > > slots). > > > > In one of my experiments, all the map tasks finish within a heartbeat > > interval of 3 seconds. I noticed that the maximum number of > > concurrently > > active map slots on my cluster never exceeds 100, and hence, the > > cluster utilization during my experiments never exceeds 50% even when > > large jobs with more than a 1000 maps are being executed. > > > > A look at the Fair Scheduler code (in particular, the assignTasks > > function) revealed the reason. > > As per my understanding, with the implementation in Hadoop 0.20.0, a > > TaskTracker is not assigned more than 1 map and 1 reduce task per > > heart beat. > > > > In my experiments, in every heart beat, each TT has 2 free map slots > > but is assigned only 1 map task, and hence, the utilization never goes > > beyond 50%. > > > > Of course, this (degenerate) case does not arise when map tasks take > > more than one 1 heart beat interval to finish. For example, I repeated > > the experiments with maps tasks taking close to 15 s to finish and > > noticed close to 100 % utilization when large jobs were executing. > > > > Why does the Fair Scheduler not assign more than one map task to a TT > > per heart beat? Is this done to spread the load uniformly across the > > cluster? > > I looked at assignTasks function in the default Hadoop scheduler > > (JobQueueTaskScheduler.java), and it does assign more than 1 map task > > per heart beat to a TT. > > > > It will be easy to change the Fair Scheduler to assign more than 1 map > > task to a TT per heart beat (I did that and achieved 100% utilization > > even with small map tasks). But I am wondering, if doing so will > > violate some fairness properties. > > > > Thanks, > > Abhishek > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
