Hi, I am seeing some strange behavior in Hadoop - I am running a small test cluster with a capacity of 18 mappers and 18 reducers. I fire a lot of jobs simultaneously and over time I have observed Hadoop is not utilizing all the 18 slots for the reducers.
And now even if I run just one job (no other jobs running), it starts less than 18 reducers. Initially it was starting all 18 but gradually it decreased. For example it started only 13 reducers for a job that I just submitted. Further, one reducer is stuck in pending state for a very long time. While all other reducers finished, one reducer was stuck in pending state for at least 20-30 minutes. The mappers seem to be doing fine. Any thoughts/suggestions what could be happening here? Cluster conf- 1) Master- also runs 4 mappers + 4 reducers 2) 2 slaves- run 7 mappers + 7 reducers I run ganglia monitoring system and I can tell you system was not overloaded at any time. Thanks, Tarandeep
