Re: Chaining multiple reducers most efficiently

2012-03-04 Thread Binglin Chang
No... Maybe in the future On Sun, Mar 4, 2012 at 10:23 PM, IGZ Nick wrote: > Hi, > > I would like to chain multiple reducers without an intervening map phase. > I looked at ChainReducer but it seems to support chains of the for M+RM*. > What I am looking for is M+R*. What would be the mos t effi

Re: reduce output compression of Terasort

2012-02-17 Thread Binglin Chang
As far as I know, TeraOutputFormat don't support compression On Fri, Feb 17, 2012 at 6:16 PM, wrote: > ** > Juwei > Is there any error messages on your TaskTracker logs related to > compression like 'Codec not found' or so ? > Regards > Bejoy K S > > From handheld, Please excuse typos. > ---

Re: Performance of direct vs indirect shuffling

2011-12-20 Thread Binglin Chang
> > One possible optimization I hope to look into next year is to change > the map output code to push the data to the local TT, which would have > configurable in-memory buffers. Have someone ever considered a general data transfer service bundled with YARN? So other applications(rather than MR)