The problem is not going away but i have a lead.... Here is a detailed diagnostic of what goes wrong: 1) 2 cluster setup on Ubuntu machines (data node and tasktracker is running on both...) 2) The reducer tries to look for a non existent file as described below: The problem happens only on my machine(slave) when it runs the reducer....When the slave executes a reducer this problem does not occur.... The map tasks seem to be running fine...
First this happens: --2009-04-27 19:54:09,901 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200904271951_0002/attempt_200904271951_0002_m_000000_0/output/file.out in any of the configured local directories --2009-04-27 19:54:46,671 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200904271951_0002/attempt_200904271951_0002_r_000000_0/output/file.out in any of the configured local directories. Then this: Repeated attempts to start another reducer fail in the same manner... So Is there a reducer specific setting that may be wrong on my machine? The task tracker seems to be off... The only inference I am able to make here is that the reducer on my machine is not able to find the mapper outputs at all... I have tried to delete the contents of mapred.local.dir and formatted the name node... So what could be wrong....?? -- View this message in context: http://www.nabble.com/Shuffle-error.-Reducers-are-not-finding-map-i-p-in-2-node-cluster....-tp23271298p23271298.html Sent from the Hadoop core-user mailing list archive at Nabble.com.