[
https://issues.apache.org/jira/browse/MAPREDUCE-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878515#action_12878515
]
Amareshwari Sriramadasu commented on MAPREDUCE-1122:
----------------------------------------------------
Users can specify Mapper/Reducer to be Java Mapper/Reducer or a command. Also,
he could specify input format, output format and partitioner for his streaming
job. The below tables summarize the mapper or reducer in use when streaming
supports both old and new api.
Note : In the tables below, NS stands for 'Not specified".
*Table 1* Mapper-in-use for given spec, when num reducers = 0:
||Mapper || InputFormat || OutputFormat || Valid conf?|| Mapper-in-use ||
|Command|NS|NS|Yes|New|
|Command|Old|NS|Yes|Old|
|Command|Old|Old|Yes|Old|
|Command|Old|New|{color:red}No{color}|
|Command|New|NS|Yes|New|
|Command|New|Old|{color:red}No{color}|
|Command|New|New|Yes|New|
|Old|NS|NS|Yes|Old|
|Old|NS|Old|Yes|Old|
|Old|Old|NS|Yes|Old|
|Old|Old|Old|Yes|Old|
|Old|-|New|{color:red}No{color}|
|Old|New|-|{color:red}No{color}|
|New|NS|NS|Yes|New|
|New|NS|New|Yes|New|
|New|New|NS|Yes|New|
|New|New|New|Yes|New|
|New|-|Old|{color:red}No{color}|
|New|Old|-|{color:red}No{color}|
*Table 2* Mapper-in-use for given spec, when num reducers != 0:
||Mapper || InputFormat || Partitioner|| Valid conf?|| Mapper-in-use ||
|Command|NS|NS|Yes|New|
|Command|Old|NS|Yes|Old|
|Command|Old|Old|Yes|Old|
|Command|Old|New|{color:red}No{color}|
|Command|New|NS|Yes|New|
|Command|New|Old|{color:red}No{color}|
|Command|New|New|Yes|New|
|Old|NS|NS|Yes|Old|
|Old|NS|Old|Yes|Old|
|Old|Old|NS|Yes|Old|
|Old|Old|Old|Yes|Old|
|Old|New|-|{color:red}No{color}|
|Old|-|New|{color:red}No{color}|
|New|NS|NS|Yes|New|
|New|NS|New|Yes|New|
|New|New|NS|Yes|New|
|New|New|New|Yes|New|
|New|Old|-|{color:red}No{color}|
|New|-|Old|{color:red}No{color}|
*Table 3* Reducer-in-use for a given spec :
|| Reducer || OutputFormat || Valid conf?|| Reducer-in-use ||
| Command | NS |Yes |New|
| Command | Old |Yes |Old |
| Command | New |Yes |New|
|Old|NS|Yes|Old|
|New|NS|Yes|New|
|Old|Old|Yes|Old|
|New|New|Yes|New|
|Old|New|{color:red}No{color}|
|New|Old|{color:red}No{color}|
> streaming with custom input format does not support the new API
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1122
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1122
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Environment: any OS
> Reporter: Keith Jackson
>
> When trying to implement a custom input format for use with streaming, I have
> found that streaming does not support the new API,
> org.apache.hadoop.mapreduce.InputFormat, but requires the old API,
> org.apache.hadoop.mapred.InputFormat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
