[
https://issues.apache.org/jira/browse/MAPREDUCE-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612716#comment-13612716
]
Harsh J commented on MAPREDUCE-5063:
------------------------------------
The need is not that highly common although it exists for certain problems.
Given that the pluggability allows for users to achieve this, I'm not sure if
it would be worth the trouble trying to maintain it upstream here as well. Look
at the join package for instance: its serving as a good example but the point
was also that folks would use it - but I don't see that happening frequently,
as people are preferring to write their own equivalents since its not highly
reusable in all scenarios.
The problem of a broadcast is simple: send it to all partitions, easy to define
and perform on flagged keys. The problem of a multi-cast is not that simple:
you want the behavior defined on a per key basis - flags get more specific.
Besides, these needs only crop up for special MR optimization cases, for which
a custom implementation should suffice.
Adding in APIs for special things is easy but we also have to consider
long-time support for it and usefulness when adding it upstream. If it can be
provided by a third party repository, then thats better and more flexible. We
can also recommend it to anyone who comes along with such a question.
> Transfering mapper output (key,value) pair to multiple reducer
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-5063
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5063
> Project: Hadoop Map/Reduce
> Issue Type: Wish
> Affects Versions: 1.0.3
> Reporter: Vikas Jadhav
> Priority: Minor
>
> Currently in Hadoop MapReduce mapper output in (key,value) form can be
> transfered to only one reducer
> Our goal is to be able transfer/shuffle (key,value) pair to multiple reducer
> Note:- we need to shuffle same pair to number of reducers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
