HI, I have to design an iterative algorithm, each iteration is a M-R cycle that calculates a parameter and has to feed it back to all the maps in the next iteration. Now the reduce procedure I need to just sum eveything from the Map procedure(Many similar size matrices) into a single matrix(of same size as each reduce ), irrespective of the key. This single matrix is the parameter I was taking about earlier. i want to know. PS This parameter MUST BE global to all map processes.
1) How do I collect all the values into one single parameter? Do I need to write it to the File system or can i keep it in memory? I feel that I WILL have to write it to the HDFS somewhere... -- View this message in context: http://www.nabble.com/Iterative-feedback-in-map-reduce....-tp22748317p22748317.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
