How about sending <0,x> to 0 and <1,x> to 1; reduce 0 can act based on the value of x?
On Wed, Mar 13, 2013 at 2:29 AM, Vikas Jadhav <vikascjadha...@gmail.com>wrote: > Hello I am not talking about custom partioner(custom partitioner is > involved but i want to write same pair for more number times) > i want it go to two reducer. > for example i have partioning attribute two dimensional > <x1,x2> > > singatue reduce > <0,0> 0 > <0,1> 1 > <1,0> 2 > <1,1> 3 > > for <1,0> it will goto reducer > for <1,null> it should goto to reducer 2 and 3 > for <0,null> it should goto reducer 0 and 1 > > On Wed, Mar 13, 2013 at 2:32 PM, Viral Bajaria <viral.baja...@gmail.com>wrote: > >> Do you want the pair to go to both reducers or do you want it to go to >> only one but in a random fashion ? >> >> AFAIK, 1st is not possible. Someone on the list can correct if I am wrong. >> 2nd is possible by just implementing your own partitioner which >> randomizes where each key goes (not sure what you gain by that). >> >> >> On Wed, Mar 13, 2013 at 1:59 AM, Vikas Jadhav >> <vikascjadha...@gmail.com>wrote: >> >>> >>> Hi >>> I am specifying requirement again with example. >>> >>> >>> >>> I have use case where i need to shufffle same (key,value) pair to >>> multiple reducers >>> >>> >>> For Example we have pair (1,"ABC") and two reducers (reducer0 and >>> reducer1) are there then >>> >>> by default this pair will go to reduce1 (cause (key % numOfReducer) = >>> (1%2) ) >>> >>> >>> how i should shuffle this pair to both reducer. >>> >>> Also I willing to change the code of hadoop framework if Necessory. >>> >>> Thank you >>> >>> On Wed, Mar 13, 2013 at 12:51 PM, feng lu <amuseme...@gmail.com> wrote: >>> >>>> Hi >>>> >>>> you can use Job#setNumReduceTasks(int tasks) method to set the number >>>> of reducer to output. >>>> >>>> >>>> On Wed, Mar 13, 2013 at 2:15 PM, Vikas Jadhav <vikascjadha...@gmail.com >>>> > wrote: >>>> >>>>> Hello, >>>>> >>>>> As by default Hadoop framework can shuffle (key,value) pair to only >>>>> one reducer >>>>> >>>>> I have use case where i need to shufffle same (key,value) pair to >>>>> multiple reducers >>>>> >>>>> Also I willing to change the code of hadoop framework if Necessory. >>>>> >>>>> >>>>> Thank you >>>>> >>>>> -- >>>>> * >>>>> * >>>>> * >>>>> >>>>> Thanx and Regards* >>>>> * Vikas Jadhav* >>>>> >>>> >>>> >>>> >>>> -- >>>> Don't Grow Old, Grow Up... :-) >>>> >>> >>> >>> >>> -- >>> * >>> * >>> * >>> >>> Thanx and Regards* >>> * Vikas Jadhav* >>> >> >> > > > -- > * > * > * > > Thanx and Regards* > * Vikas Jadhav* >
