I was not precise when describing my cluster. I have 4 slave nodes and a separate master node. The master has ResourceManager role (along with JobHistory role) and the rest have NodeManager roles. If this really is an ApplicationMaster, is it possible to schedule it on the master node? This single waiting map task is doubling my execution time.
Pozdrawiam / Regards / Med venlig hilsen Tomasz Guziałek 2014-07-08 18:42 GMT+02:00 Adam Kawa <[email protected]>: > Is not your MapReduce AppMaster occupying one slot? > > Sent from my iPhone > > > On 8 jul 2014, at 13:01, Tomasz Guziałek <[email protected]> > wrote: > > > > Hello all, > > > > I am running a 4-nodes CDH5 cluster on Amazon EC2 . The instances used > are m1.large, so I have 4 cores (2 core x 2 unit) per node. My HBase table > has 8 regions, so I expected at least 8 (if not 16) mapper tasks to run > simultaneously. However, only 7 are running and 1 is waiting for an empty > slot. Why this surprising number came up? I have checked that the regions > are equally distributed on the region servers (2 per node). > > > > My properties in the job: > > Configuration mapReduceConfiguration = HBaseConfiguration.create(); > > mapReduceConfiguration.set("hbase.client.max.perregion.tasks", "4"); > > mapReduceConfiguration.set("mapreduce.tasktracker.map.tasks.maximum", > "16"); > > > > My properties in the CDH: > > yarn.scheduler.minimum-allocation-vcores = 1 > > yarn.scheduler.maximum-allocation-vcores = 4 > > > > Do I miss some property? Please share your experience. > > > > Best regards > > Tomasz >
