Re: Is there a way to insure that different jobs have the same number of reducers

Exact same bucket is possible, exact same machine (if that is what you had in mind) probably not. The partitioner breaks the data up for the reducers, so if they map to the same partition they will be done by the same reducer. If you can partition the data such that the output of one reducer partit

Re: Reduce method called same key twice

ondary > fields to group on) it works as it should. > > --Aaron > > > --------- > From: Trevor Adams [mailto:trevorad...@gmail.com] > Sent: Wednesday, June 29, 2011 11:34 AM > To: mapreduce-user@hadoop.apache

Re: Reduce method called same key twice

y. > > --Aaron > > > ----- > From: Trevor Adams [mailto:trevorad...@gmail.com] > Sent: Wednesday, June 29, 2011 10:00 AM > To: mapreduce-user@hadoop.apache.org > Subject: Reduce method called same key twice > > So I

Reduce method called same key twice

So I have a custom Key which is used for a join. It contains two fields, a boolean (is primary key) and an int (key). Hashcode only looks at the key field, so that it gets sent to the same reducer. Compare places the pkey at the top of the list (if sorted using compare). This works nicely, except t