On Thu, 27 Mar 2008, Natarajan, Senthil wrote:
> Hi, > I have small Hadoop cluster, one master and three slaves. > When I try the example wordcount on one of our log file (size ~350 MB) > > Map runs fine but reduce always hangs (sometime around 19%,60% ...) after > very long time it finishes. > I am seeing this error > Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out This error occurs when the reducer fails to fetch map-task-output from 5 unique map tasks. Before considering an attempt as failed the reducer tries to fetch the map output for 7 times in 5 mins (default config). In case of the job failure check the following 1. Is this problem common in all the reducers? 2. Are the map tasks same across all the reducers for which the failure is reported? 3. Is there atleast one map task whose output is successfully fetched? If the job becomes successful then there might be some problem with the reducer. Amar > In the log I am seeing this > INFO org.apache.hadoop.mapred.TaskTracker: task_200803261535_0001_r_000000_0 > 0.18333334% reduce > copy (11 of 20 at 0.02 MB/s) > > > Do you know what might be the problem. > Thanks, > Senthil > >
