[
https://issues.apache.org/jira/browse/TEZ-698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924591#comment-13924591
]
Mohammad Kamrul Islam commented on TEZ-698:
-------------------------------------------
I find out the following lines of code in WordCount to be combined into one
place for MRInput.
The possible hosted class can be: InputDescriptor
{noformat}
byte[] mapPayload = MRHelpers.createUserPayloadFromConf(mapStageConf);
byte[] mapInputPayload =
MRHelpers.createMRInputPayloadWithGrouping(mapPayload,
TextInputFormat.class.getName());
InputDescriptor id = new InputDescriptor(MRInput.class.getName()).
setUserPayload(mapInputPayload);
{noformat}
For Output,
{noformat}
byte[] finalReducePayload =
MRHelpers.createUserPayloadFromConf(finalReduceConf);
OutputDescriptor od = new OutputDescriptor(MROutput.class.getName())
.setUserPayload(finalReducePayload);
{noformat}
I'm not sure that's all for MRInput and MROutput. Also didn't find a lot about
ShuffleInput/SortedOutput in wordcount example.
> Make it easy to create and configure
> MRInput/MROutput/ShuffleInput/SortedOutput
> -------------------------------------------------------------------------------
>
> Key: TEZ-698
> URL: https://issues.apache.org/jira/browse/TEZ-698
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
>
> We have moved away from MR and its not necessary for anyone to write mappers
> and reducers or to configure them. But MR input and output and Shuffle
> related inputs/outputs. Currently we have to invoke a host of methods to
> configure them. If we can have a single API to make these configs then it
> would really help. Secondly for IO pairs like ShuffleInput/SortedOutput,
> their configs are related (KV types e.g.) So it maybe useful to have a
> combined API that generates configs for both in a single API.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
