Mapper Input
7 vim
2 emacs
9 firefox
Mapper output ( new key = Integer.MAX_VALUE - key value)
2147483640 vim
2147483645 emacs
2147483636 firefox
Note :Integer.MAX_VALUE is 2147483647 (which would be 2^31 - 1)
Hadoop will sort the records for you.
If you are using single reducer, reducer input would be
2147483636 firefox
2147483640 vim
2147483645 emacs
reducer output ( this time subtract key from Integer.MAX_VALUE to get
back original value)
9 firefox
7 vim
2 emacs
On 11/06/2013 13:05, Pedro Sá da Costa wrote:
Even with your answer I can't see how can I sort the data in reverse
order. I forgot to mention that, the output result is produced by one
reduce task. This means that, at any point of the execution of the
job, the data must be grouped and sorted in descendent order.
On 11 June 2013 13:57, Bhasker Allene <allene.bhas...@gmail.com
<mailto:allene.bhas...@gmail.com>> wrote:
One way to approach is emit |Integer.MAX_VALUE - your key ||as
output of mapper.|
Example
Mapper input
7 vim
2 emacs
9 firefox
Mapper output
(Integer.MAX_VALUE - 7) vim
(Integer.MAX_VALUE - 2) emacs
(Integer.MAX_VALUE - 9) firefox
If you need secondary sorting on second part, you have to use
composite key and write your own petitioner, comparator.
Regards,
Bhasker
On 11/06/2013 11:10, Pedro Sá da Costa wrote:
I created a MapReduce job example that that uses the sort
mechanism of hadoop to sort a file by the key in ascending order.
This is an example of the data:
7 vim
2 emacs
9 firefox
At the end, I get the result:
2 emacs
7 vim
9 firefox
Now I want to sort in reverse order, for the result be:
9 firefox
7 vim
2 emacs
How can I sort a file with pairs <Key Value> in reverse order?
--
Best regards,
--
Thanks & Regards,
Bhasker Allene
--
Best regards,
--
Thanks & Regards,
Bhasker Allene
