I am seeing an odd mix of errors in a job we have running on a particular
cluster of machines.
Has anyone seen this before and what is actually the problem?
We are running linux (Centos51, on 8 way xeons, with all disks under raid 5)
and GigE switches between the machines.
The namenode machine does not run a datanode or a tasktracker and is
essentially idle.
Thanks!
2008-07-18 09:25:43,626 WARN org.apache.hadoop.mapred.TaskTracker: Error
running child
java.io.IOException: Filesystem closed
at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:158)
at org.apache.hadoop.dfs.DFSClient.access$500(DFSClient.java:58)
at
org.apache.hadoop.dfs.DFSClient$DFSInputStream.close(DFSClient.java:1095)
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at
org.apache.hadoop.mapred.LineRecordReader$LineReader.close(LineRecordReader.java:97)
at
org.apache.hadoop.mapred.LineRecordReader.close(LineRecordReader.java:277)
at
org.apache.hadoop.mapred.KeyValueLineRecordReader.close(KeyValueLineRecordReader.java:113)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.close(MapTask.java:155)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:212)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)
and
2008-07-18 04:26:41,056 WARN org.apache.hadoop.mapred.TaskTracker: Error
running child
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
task_200807141819_0007_m_001981_1/spill0.out in any of the configured local
directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:359)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
at
org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:77)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:464)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:713)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:209)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)
--
Jason Venner
Attributor - Program the Web <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers and coding wizards, contact if
interested