+1 Good performance improvement. Nice work…
On Sat, Sep 6, 2014 at 6:05 AM, Chris Douglas <[email protected]> wrote: > +1 > > The change to the existing code is very limited and the perf is > impressive. -C > > On Fri, Sep 5, 2014 at 4:58 PM, Todd Lipcon <[email protected]> wrote: > > Hi all, > > > > As I've reported recently [1], work on the MAPREDUCE-2841 branch has > > progressed well and the development team working on it feels that it is > > ready to be merged into trunk. > > > > For those not familiar with the JIRA (it's a bit lengthy to read from > start > > to finish!) the goal of this work is to build a native implementation of > > the map-side sort code. The native implementation's primary advantage is > > its speed: for example, terasort is 30% faster on a wall-clock basis and > > 60% faster on a resource consumption basis. For clusters which make heavy > > use of MapReduce, this is a substantial improvement to their efficiency. > > Users may enable the feature by switching a single configuration flag, > and > > it will fall back to the original implementation in cases where the > native > > code doesn't support the configured features/types. > > > > The new work is entirely pluggable and off-by-default to mitigate risk. > The > > merge patch itself does not modify even a single line of existing code: > all > > necessary plug-points have already been committed to trunk for some time. > > > > Though we do not yet have a full +1 precommit Jenkins run on the JIRA, > > there are only a few small nits to fix before merge, so I figured that we > > could start the vote in parallel. Of course we will not merge until it > has > > a positive precommit run. > > > > Though this branch is a new contribution to the Apache repository, it > > represents work done over several years by a large community of > developers > > including the following: > > > > Binglin Chang > > Yang Dong > > Sean Zhong > > Manu Zhang > > Zhongliang Zhu > > Vincent Wang > > Yan Dong > > Cheng Lian > > Xusen Yin > > Fangqin Dai > > Jiang Weihua > > Gansha Wu > > Avik Dey > > > > The vote will run for 7 days, ending Friday 9/12 EOD PST. > > > > I'll start the voting with my own +1. > > > > -Todd > > > > [1] > > > http://search-hadoop.com/m/09oay13EwlV/native+task+progress&subj=Native+task+branch+progress > -- Thanks Devaraj K
