Re: how to handle large volume reduce input value in mapreduce program?

2009-10-05 Thread Aaron Kimball
HB, >From a scalability perspective, you will always have a finite limit of RAM available. Without knowing much about HBase, I can't tell whether this is the only way you can accomplish your goal or not. But a basic maxim to follow is, "Using O(N) space in your reducer is guaranteed to overflow fo

Re: Does Using MultipleTextOutputFormat Require the Deprecated API?

2009-10-05 Thread Aaron Kimball
Geoffry, The new API comes with a related OF, called MultipleOutputs (o.a.h.mapreduce.lib.output.MultipleOutputs). You may want to look into using this instead. - Aaron On Tue, Sep 29, 2009 at 4:44 PM, Geoffry Roberts wrote: > All, > > What I want to do is output from my reducer multiple files