RE: Reducer not firing

2012-04-16 Thread Devaraj k
Hi Arko, What is value of 'no_of_reduce_tasks'? If no of reduce tasks are 0, then the map task will directly write map output into the Job output path. Thanks Devaraj From: Arko Provo Mukherjee [arkoprovomukher...@gmail.com] Sent: Tuesday, April

Re: map and reduce with different value classes

2012-04-16 Thread Bryan Yeung
Oh no! I just figured it out :-/ It's actually dying on the Combine step, with an error about the Reduce class (because the WordCount example uses the Reduce class as both the reducer and combiner). This makes sense now. Sorry for the silly question, and thanks for the help! Bryan On Mon, Apr

Re: map and reduce with different value classes

2012-04-16 Thread Bejoy Ks
Hi Bryan Can you post in the error stack trace? Regards Bejoy KS On Tue, Apr 17, 2012 at 8:41 AM, Bryan Yeung wrote: > Hello Bejoy, > > Thanks for your reply. > > Isn't that exactly what I've done with my modifications to > WordCount.java?  Could you have a look at the diff I supplied and/

Re: map and reduce with different value classes

2012-04-16 Thread Bryan Yeung
Hello Bejoy, Thanks for your reply. Isn't that exactly what I've done with my modifications to WordCount.java? Could you have a look at the diff I supplied and/or the WordCount.java file I attached and tell me how I've deviated from what you say below? Thanks, Bryan On Mon, Apr 16, 2012 at 11

Re: map and reduce with different value classes

2012-04-16 Thread Bejoy Ks
HI Bryan      You can set different key and value types with the following steps - ensure that the map output key value type is the reducer input key value type - specify it on your Driver Class as //set map output key value types job.setMapOutputKeyClass(theClass)   job.setMapOutputValueClass(the

map and reduce with different value classes

2012-04-16 Thread Bryan Yeung
Hello Everyone, I'm relatively new to hadoop mapreduce and I'm trying to get this simple modification to the WordCount example to work. I'm using hadoop-1.0.2, and I've included both a convenient diff and also attached my new WordCount.java file. The thing I am trying to achieve is to have the v