Re: Reduce method called same key twice

2011-06-29 Thread Trevor Adams
primary key, and then if they are the > same compares the int part of the key. > > --Aaron > > > - > From: Trevor Adams [mailto:trevorad...@gmail.com] > Sent: Wednesday, June 29, 2011 10:00 AM > To: ma

RE: Reduce method called same key twice

2011-06-29 Thread Aaron Baff
Adams [mailto:trevorad...@gmail.com] Sent: Wednesday, June 29, 2011 11:34 AM To: mapreduce-user@hadoop.apache.org Subject: Re: Reduce method called same key twice So, that kind of makes sense but why would it not group the other values then? There are a bunch of the exact same key (only 1 primary

Re: Reduce method called same key twice

2011-06-29 Thread Trevor Adams
y. > > --Aaron > > > - > From: Trevor Adams [mailto:trevorad...@gmail.com] > Sent: Wednesday, June 29, 2011 10:00 AM > To: mapreduce-user@hadoop.apache.org > Subject: Reduce method called same key twice > > So I

RE: Reduce method called same key twice

2011-06-29 Thread Aaron Baff
[mailto:trevorad...@gmail.com] Sent: Wednesday, June 29, 2011 10:00 AM To: mapreduce-user@hadoop.apache.org Subject: Reduce method called same key twice So I have a custom Key which is used for a join. It contains two fields, a boolean (is primary key) and an int (key). Hashcode only looks at the key

Reduce method called same key twice

2011-06-29 Thread Trevor Adams
So I have a custom Key which is used for a join. It contains two fields, a boolean (is primary key) and an int (key). Hashcode only looks at the key field, so that it gets sent to the same reducer. Compare places the pkey at the top of the list (if sorted using compare). This works nicely, except t