Hi Harsh, thanks for your answer!. The cluster is homogeneus, every node has the same amount of cores and memory and is equally reachable in the network. The data is generated specifically for each run. I mean, I write the input data in 4 nodes for one run and in 7 nodes for another. So the input file will be replicated in 4 nodes when running the map reduce with 4 nodes, and in 7 nodes when running it with 7. I don't know if speculatives maps are on, I'll check it. One thing I observed is that reduces begin before all maps have finished. Let me check also if the difference is on the map side or in the reduce. I believe it's balanced, both are slower when adding more nodes, but i'll confirm that.
I would appreciate any other comment, thanks again On 21 June 2011 13:33, Harsh J <ha...@cloudera.com> wrote: > Alberto, > > Please add more practical-related info like if your cluster is > homogenous, if the number of maps and reduces in both runs are > consistent (i.e., same data and same amount of reducers on 4 vs. 7?), > and if map speculatives are on. Also, do you notice difference of time > for a single map task across the two runs? Or is the difference on the > reduce task side? > > On Tue, Jun 21, 2011 at 8:33 PM, Alberto Andreotti > <albertoandreo...@gmail.com> wrote: > > Hello, > > > > I'm working with an application to calculate the temperatures of a > squared > > board. I divide the board in a mesh, and represent the board as a list of > > (key, value) pairs with a key being the linear position of a cell within > the > > mesh, and the value its temperature. > > I distribute the data during the map and calculate the temperature for > next > > step in the reduce. You can see a more detailed explanation here, > > > > > http://code.google.com/p/heat-transfer/source/browse/trunk/informe/Informe.pdf > > > > but the basic idea is the one I have just mentioned. > > The funny thing is that the more nodes I add the slower it runs!. With 7 > > nodes it takes 16 minutes, but with 4 nodes it takes only 8 minutes. > > You can see the code in file HeatTransfer.java which is found here, > > > > > http://code.google.com/p/heat-transfer/source/browse/#svn%2Ftrunk%2Ffine%253Fstate%253Dclosed > > > > thanks in advance! > > > > Alberto. > > -- > > José Pablo Alberto Andreotti. > > Tel: 54 351 4730292 > > Móvil: 54351156526363. > > MSN: albertoandreo...@gmail.com > > Skype: andreottialberto > > > > > > -- > Harsh J > -- José Pablo Alberto Andreotti. Tel: 54 351 4730292 Móvil: 54351156526363. MSN: albertoandreo...@gmail.com Skype: andreottialberto
