Re: Stupid questions about combiners in ...hadoop.mapreduce

2011-05-23 Thread Chris Douglas
On Mon, May 23, 2011 at 11:32 AM, Mike Spreitzer wrote: > What happens if one invocation of a combiner outputs more than one value? > > What happens if an output key is different from the input key? The combiner is responsible for maintaining the sort order (and partitioning) effected prior to th

Re: Stupid questions about combiners in ...hadoop.mapreduce

2011-05-23 Thread James Warren
the same class)? > > Even for question 3, I did not exactly see an answer to "what happens" --- > only a statement that I should not exercise that case. > > Thanks, > Mike > > > > From:Ted Yu > To: mapreduce-user@hadoop.apache.org &

Re: Stupid questions about combiners in ...hadoop.mapreduce

2011-05-23 Thread Mike Spreitzer
of the same class)? Even for question 3, I did not exactly see an answer to "what happens" --- only a statement that I should not exercise that case. Thanks, Mike From: Ted Yu To: mapreduce-user@hadoop.apache.org Date: 05/23/2011 03:04 PM Subject: Re: Stupid quest

Re: Stupid questions about combiners in ...hadoop.mapreduce

2011-05-23 Thread Ted Yu
Questions 2 and 3 can be answered relatively easily: Remember, the output of the combiner is going to be consumed by the reducer. So the output key/vlaue classes of the combiner have to align with the input key/vlaue classes of the reducer. On Mon, May 23, 2011 at 11:32 AM, Mike Spreitzer wrote:

Stupid questions about combiners in ...hadoop.mapreduce

2011-05-23 Thread Mike Spreitzer
In general, the Java interfaces say that one invocation of a combiner (technically, a Class) can output multiple (key,value) pairs. So: What happens if one invocation of a combiner outputs more than one value? What happens if an output key is different from the input key? What happens if an o