[
https://issues.apache.org/jira/browse/MAPREDUCE-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Oaks updated MAPREDUCE-7338:
----------------------------------
Affects Version/s: 3.1.3
> ArrayIndexOutOfBoundsException when sort buffer wraps incorrectly
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-7338
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7338
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 3.1.3
> Reporter: Scott Oaks
> Priority: Minor
>
> Exception generated when writing output from map task:
> {noformat}
> Error: java.lang.ArrayIndexOutOfBoundsException at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1469)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1365)
> at java.io.DataOutputStream.writeByte(DataOutputStream.java:153) at
> org.apache.hadoop.io.WritableUtils.writeVLong(WritableUtils.java:273) at
> org.apache.hadoop.io.WritableUtils.writeVInt(WritableUtils.java:253) at
> org.apache.hadoop.io.Text.write(Text.java:330) at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:98)
> at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:82)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1163)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:727)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> {noformat}
> This happens in our case when the io.sort.mb is set to 1750; if we reduce the
> size of the buffer it wraps correctly and no exception is generated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
