Hello, I have an intensive job running across 5 machines. During the map stage, each map emits 200 records, so effectively for 50,000,000 input reords, the map creates 200*50e6 records.
However, after a long time, I see two trackers are blacklisted Caused by: java.lang.RuntimeException: Could not obtain block: blk_-5964245027287878843_92134 file=/tmp/8a5bc814-b4ff-4641-bc3a-abfeda9e7e33.mapfile/index at org.saptarshiguha.rhipe.hadoop.RHMR$RHMRCombiner.configure(RHMR.java:314) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:1110) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:989) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:401) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:886) The map file is very much present in that location and other trackers could read it. What could be the reason for these two machines not being able to read it? Now that they are blacklisted, how can I add them back to the computation (a Saptarshi Guha