Hi all,
I just try to use MultithreadedMapRunner for wordcount with 2 threads in one
maptask, but each maptask gets the following exception, and the job failed
eventually. But when running with only 1 thread, there is no such error.
java.lang.ArrayIndexOutOfBoundsException
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1122)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.Text.write(Text.java:282)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:906)
at
org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:518)
at WordCount$Map.map(WordCount.java:25)
at WordCount$Map.map(WordCount.java:1)
at
org.apache.hadoop.mapred.lib.MultithreadedMapRunner$MapperInvokeRunable.run(MultithreadedMapRunner.java:229)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Version: hadoop-0.20.1
OS: CentOS 5.2 (64bit)
Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)
with thanks, Leitao
