Looking for correct way to implements WritableComparable in Hadoop-0.17

2009-06-20 Thread Kunsheng Chen
Hello everyone, I am writing my own Comparator inherits from WritableComparable. I got the folliowing code from Hadoop definitive guide, which is not working at all, it reminds me WritableComparable does not take parameter. The book might be using Hadoop-0.21 I also tried the old method for

Re: Anyway to sort keys before Reduce function in Hadoop ?

2009-06-17 Thread Kunsheng Chen
(grep for combine) You may also find these videos useful: http://www.cloudera.com/hadoop-training-mapreduce-hdfs http://www.cloudera.com/hadoop-training-programming-with-hadoop Hope this helps.  Let me know if I misunderstood your question. Alex On Mon, Jun 15, 2009 at 4:22 PM, Kunsheng

Re: Could I collect results from Map-Reduce then output myself ?

2009-06-15 Thread Kunsheng Chen
, discarding the data that doesn't meet your criterion. The input path to your second MR is the output path from the first one. - Aaron On Sun, Jun 14, 2009 at 4:02 PM, Kunsheng Chen ke...@yahoo.com wrote: Hi everyone, I am doing a map-reduce program, it is working good. Now I am

Could I output results into multiple files in hadoop ?

2009-06-14 Thread Kunsheng Chen
Hi, everyone, Is there any way to output the results in Reduce to different files ? thanks in advance, -Kun

Re: what is the efficient way to implement InputFormat

2009-06-01 Thread Kunsheng Chen
Hi Sun, Sounds you are dealing with I met before. The format of my file is something like this, there is only a space between each element source destination degree timestamp. for example: http://www.google.com http://something.net 3 1234555 The source is my key for map and reduce, I

Runtime error for wordcount-like example.

2009-05-24 Thread Kunsheng Chen
Hello everyone, I am getting some issues when compiling a map-reduce program similar to wordcount example. I followed the instruction in the tutorial and succeeded to compile a jar file. The problem won't appear until I run it as below: had...@lobo:/osr/Projects/anansi/mrjobs/src$