MapFileOutputFormat and SequenceFileOutputFormat use incorrect key/value
classes in map/reduce tasks
----------------------------------------------------------------------------------------------------
Key: HADOOP-920
URL: https://issues.apache.org/jira/browse/HADOOP-920
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.11.0
Reporter: Andrzej Bialecki
Fix For: 0.11.0
Let's assume a job uses different key/value class for the output of map tasks
and for the final output of reduce tasks.
When executing map tasks classes returned from JobConf.getMapOutputKeyClass() /
getMapOutputValueClass() should be used, and when executing reduce tasks
classes returned from JobConf.gtOutputKeyClass() / getOutputValueClass() should
be used.
Currently both map and reduce tasks will use
getMapOutputKeyClass/getMapOutputValueClass when using MapFileOutputFormat, or
they will always use getOutputKeyClassgetOutputValueClass when using
SequenceFileOutputFormat. This causes exceptions, because Mapper / Reducer
implementations will output different key/value classes than expected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
