Hello Tom, Would like to apply some rules To CAT_A, then use the output of CAT_A to reduce CAT_B. I'd rather not run two JOBS, so perhaps I need two reducers?
First Reducer processes CAT_A, then when complete second reducer does CAT_B? I suppose this would accomplish the same thing? -----Original Message----- From: Tom White [mailto:[email protected]] Sent: Thursday, January 22, 2009 10:41 AM To: [email protected] Subject: Re: Set the Order of the Keys in Reduce Hi Brian, The CAT_A and CAT_B keys will be processed by different reducer instances, so they run independently and may run in any order. What's the output that you're trying to get? Cheers, Tom On Thu, Jan 22, 2009 at 3:25 PM, Brian MacKay <[email protected]> wrote: > Hello, > > > > Any tips would be greatly appreciated. > > > > Is there a way to set the order of the keys in reduce as shown below, no > matter what order the collection in MAP occurs in. > > > > Thanks, Brian > > > > > > public void map(WritableComparable key, Text values, > > OutputCollector<Text, Text> output, Reporter reporter) > throws IOException { > > > > //collect many CAT_A and CAT_B in random order > > output.collect(CAT_A, details); > > output.collect(CAT_B, details); > > > > } > > > > > > > > public void reduce(Text key, Iterator<Text> values, > > OutputCollector<Text, Text> output, Reporter > reporter) throws IOException { > > > > //always reduce CAT_A first, then reduce CAT_B > > > > } > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this message in error, please contact the sender and delete the material > from any computer. > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this message in error, please contact the sender and delete the material from any computer.
