Do you want the pair to go to both reducers or do you want it to go to only one but in a random fashion ?
AFAIK, 1st is not possible. Someone on the list can correct if I am wrong. 2nd is possible by just implementing your own partitioner which randomizes where each key goes (not sure what you gain by that). On Wed, Mar 13, 2013 at 1:59 AM, Vikas Jadhav <vikascjadha...@gmail.com>wrote: > > Hi > I am specifying requirement again with example. > > > > I have use case where i need to shufffle same (key,value) pair to multiple > reducers > > > For Example we have pair (1,"ABC") and two reducers (reducer0 and > reducer1) are there then > > by default this pair will go to reduce1 (cause (key % numOfReducer) = > (1%2) ) > > > how i should shuffle this pair to both reducer. > > Also I willing to change the code of hadoop framework if Necessory. > > Thank you > > On Wed, Mar 13, 2013 at 12:51 PM, feng lu <amuseme...@gmail.com> wrote: > >> Hi >> >> you can use Job#setNumReduceTasks(int tasks) method to set the number of >> reducer to output. >> >> >> On Wed, Mar 13, 2013 at 2:15 PM, Vikas Jadhav >> <vikascjadha...@gmail.com>wrote: >> >>> Hello, >>> >>> As by default Hadoop framework can shuffle (key,value) pair to only one >>> reducer >>> >>> I have use case where i need to shufffle same (key,value) pair to >>> multiple reducers >>> >>> Also I willing to change the code of hadoop framework if Necessory. >>> >>> >>> Thank you >>> >>> -- >>> * >>> * >>> * >>> >>> Thanx and Regards* >>> * Vikas Jadhav* >>> >> >> >> >> -- >> Don't Grow Old, Grow Up... :-) >> > > > > -- > * > * > * > > Thanx and Regards* > * Vikas Jadhav* >
