Output a constant key in the map function.
On 1/15/08 9:31 PM, "Vadim Zaliva" <[EMAIL PROTECTED]> wrote: > On Jan 15, 2008, at 17:56, Peter W. wrote: > > That would output last 10 values for each key. I need > to do this across all the keys in the set. > > Vadim > >> Hello, >> >> Try using Java collection. >> >> untested code follows... >> >> public static class R extends MapReduceBase implements Reducer >> { >> public void reduce(WritableComparable wc,Iterator it, >> OutputCollector out,Reporter r)throws IOException >> { >> Stack s=new Stack(); >> int cnt=0; >> >> // everything >> while(it.hasNext()) >> { >> s.push(((IntWritable)it.next()).get()); >> } >> >> // last 10 >> while((!s.empty())&&(cnt<=10)) >> { >> out.collect(wc,(IntWritable)s.pop()); >> cnt++; >> } >> } >> } >> >> Good Luck, >> >> Peter W. >> >> Vadim Zaliva wrote: >> >>> >>> Rui Shi wrote: >>> >>>> As far as I understand, let mapper produce top N records is not >>>> working >>>> as each mapper only has partial knowledge of the data, which will >>>> not lead to >>>> global optimal... I think your mapper needs to output all records >>>> (combined) and let the reducer to pick the top N values. >>> >>> the question remains, how to return, say, last 10 records from >>> Reducer. >>> I need to know when last record is processed. >>> >>> Vadim >> >