Re: Automatic line number in reducer output

2011-06-10 Thread Shi Yu
Yes, it works perfectly. Actually didn't realize the flexibility to employ different classes in combiner and reducer. In that case it would have a three layer architecture, think that would be interesting and useful. Shi On 6/10/2011 9:31 AM, Robert Evans wrote: In this case you probably want

Re: Automatic line number in reducer output

2011-06-10 Thread Robert Evans
In this case you probably want two different classes. You can have the base Reducer class that adds in the line count, and then subclass it for the combiner, that sets a flag to not output the line numbers. --Bobby On 6/9/11 12:57 PM, "Shi Yu" wrote: Hi, Thanks for the reply. The line coun

Re: Automatic line number in reducer output

2011-06-09 Thread Shi Yu
Hi, Thanks for the reply. The line count in new API works fine now, it was a bug in my code. In new API, Iterator is changed to Iterable, but I didn't pay attention to that and was still using Iterator and hasNext(), Next() method. Surprisingly, the wrong code still ran and got output, but

Re: Automatic line number in reducer output

2011-06-09 Thread Robert Evans
What exactly is linecount being output as in the new APIs? --Bobby On 6/7/11 11:21 AM, "Shi Yu" wrote: Hi, I am wondering is there any built-in function to automatically add a self-increment line number in reducer output (like the relation DB auto-key). I have this problem because in 0.19.2 A

Automatic line number in reducer output

2011-06-07 Thread Shi Yu
Hi, I am wondering is there any built-in function to automatically add a self-increment line number in reducer output (like the relation DB auto-key). I have this problem because in 0.19.2 API, I used a variable linecount increasing in the reducer like: public static class Reduce extends