Check out the EM example in nltk: http://code.google.com/p/nltk/source/browse/trunk/nltk/nltk_contrib/hadoop/EM/runStreaming.py
On Fri, Mar 27, 2009 at 5:19 PM, Sid123 <[email protected]> wrote: > > HI, > I have to design an iterative algorithm, each iteration is a M-R cycle that > calculates a parameter and has to feed it back to all the maps in the next > iteration. > Now the reduce procedure I need to just sum eveything from the Map > procedure(Many similar size matrices) into a single matrix(of same size as > each reduce ), irrespective of the key. This single matrix is the parameter > I was taking about earlier. > i want to know. PS This parameter MUST BE global to all map processes. > > 1) How do I collect all the values into one single parameter? Do I need to > write it to the File system or can i keep it in memory? I feel that I WILL > have to write it to the HDFS somewhere... > -- > View this message in context: > http://www.nabble.com/Iterative-feedback-in-map-reduce....-tp22748317p22748317.html > Sent from the Hadoop core-user mailing list archive at Nabble.com. > > -- Peter N. Skomoroch 617.285.8348 http://www.datawrangling.com http://delicious.com/pskomoroch http://twitter.com/peteskomoroch
