Currently i am doing some experiments with WordCount example ,so considering this TestMerge.java ,it seems i need to change WordCount example and add MapOutputCopier and etc like what Mariappan done in TestMerge to nullify sort, right ?
Samaneh On Tue, Apr 23, 2013 at 10:49 AM, Harsh J <[email protected]> wrote: > Yes as Mariappan has already pointed out; the sort is pluggable so the > plug-in can also nullify it. See [1] for an example implementation of > the MapOutputCollector interface (class MapOutputCopier) to start > with. Per Mariappan, this does not do any sorting, but only merges. > > [1] - > http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMerge.java > > On Tue, Apr 23, 2013 at 2:00 PM, Samaneh Shokuhi > <[email protected]> wrote: > > Hi Harsh, > > That is nice if branch-2 included that .Just one thing i want to be sure > > about that .In this patch MAPREDUCE-4807 (branch-2) , sort in mapper is > > avoided ? Becouse i need a version of hadoop without sorting part in > mapper > > and thats the reason i wanted to modify MapTask to disable the mapper > sort > > .Do you think branch-2 is appropriate version for me ? > > > > Samaneh > > > > > > On Tue, Apr 23, 2013 at 10:01 AM, Harsh J <[email protected]> wrote: > > > >> Note that the Reducer also does a merge sort over the acquired map > >> data; but am not sure if you looked there as well. > >> > >> Mariappan's pointed JIRA (MAPREDUCE-4807) is already available in the > >> 2.0.3+ releases out today and in the current branch-2. It would be > >> simpler to reuse that than do these modifications. > >> > >> On Sun, Apr 21, 2013 at 5:24 PM, Samaneh Shokuhi > >> <[email protected]> wrote: > >> > Hi All, > >> > I have modified the MapTask.java file and tried to disable sort part > for > >> > some reasons . I have removed this line > >> > *sorter.sort(MapOutputBuffer.this, kvstart, endPosition, reporter); > *in > >> > sortAndSpill method but no changes seen in result. I have expected to > get > >> > unsorted keys as mapper output but was not like that. > >> > > >> > My question is did i perform right action to disable sort or something > >> else > >> > needs to be done ? > >> > > >> > Samaneh > >> > >> > >> > >> -- > >> Harsh J > >> > > > > -- > Harsh J >
