Re: Tips on sorting using Hadoop

2008-09-24 Thread bz
on sorting using Hadoop I would recommend that run map/reduce twice. /Edward On Sat, Sep 13, 2008 at 5:58 AM, Tenaali Ram [EMAIL PROTECTED] wrote: Hi, I want to sort my records ( consisting of string, int, float) using Hadoop. One way I have found is to set number of reducers = 1

Re: Tips on sorting using Hadoop

2008-09-20 Thread lohit
: Re: Tips on sorting using Hadoop I would recommend that run map/reduce twice. /Edward On Sat, Sep 13, 2008 at 5:58 AM, Tenaali Ram [EMAIL PROTECTED] wrote: Hi, I want to sort my records ( consisting of string, int, float) using Hadoop. One way I have found is to set number of reducers = 1

Re: Tips on sorting using Hadoop

2008-09-20 Thread Owen O'Malley
On Sat, Sep 20, 2008 at 11:12 AM, lohit [EMAIL PROTECTED] wrote: To do total order sorting, you have to make your partition function split the keyspace equally in order among the number of reducers. A library to do this was checked in yesterday. See HADOOP-3019. -- Owen

Tips on sorting using Hadoop

2008-09-12 Thread Tenaali Ram
Hi, I want to sort my records ( consisting of string, int, float) using Hadoop. One way I have found is to set number of reducers = 1, but this would mean all the records go to 1 reducer and it won't be optimized. Can anyone point me to some better way to do sorting using Hadoop ? Thanks,