This is fine. 40 may be ending, while the next 40 may be starting. The tasks will run a cleanup operation at their end (and be in such a 'status'), during which the TaskTracker is allowed to schedule the next wave of maps it needs. In these moments, it may appear as if 80 concurrent tasks may be running. You can verify this by looking at the state of the 80 'running' tasks. 40~ of them would be initializing while 40~ of them would be cleaning up.
Reducer slots are not abused by the Mappers. On Tue, Jan 25, 2011 at 6:33 PM, exception <[email protected]> wrote: > Hi, > > > > My cluster contains 5 DataNodes, each with 8 map slots and 2 reduce slots. > So there are up to 40 slots in my cluster and 40 tasks can run in parallel. > > But when running a particular job, I have noticed that 80 tasks running in > parallel. The cluster looks fine when running other jobs. > > > -- Harsh J www.harshj.com
