Oh and sorry, I was just seeing that you use the SSSP vertex. In case of using SSSP, this is perfectly normal, I don't know if you read the Pregel paper, but it has a great "analogy" of how the algorithm works: "resulting in a wavefront of updates through the graph". Technically speaking, it is sending more messages if more vertices are active. Where in the first superstep there is only the startvertex active, in the next superstep it's neighbours, in the overnext the neighbours of the neighbours and so on. So you end up with a steadily increasing number of active vertices (in the breadth) and thus more messages.
2013/2/20 Suraj Menon <[email protected]> > Francisco is helping us experiment new patches on our trunk in his cluster > and would give his feedback. > > On Wed, Feb 20, 2013 at 11:26 AM, Thomas Jungblut < > [email protected] > > wrote: > > > Just analysed this: > > https://issues.apache.org/jira/browse/HAMA-704#comment-13582292 > > > > I'm seeing no memory leak with pagerank, this time it must be in your > code > > :-P > > > > 2013/2/20 Thomas Jungblut <[email protected]> > > > > > That looks indeed like a leak. The problem is that this could be > > anywhere, > > > even in your user code. > > > Will brew something together to reproduce this... > > > > > > > > > 2013/2/20 Francisco Sanches <[email protected]> > > > > > >> Hi Thomas, > > >> > > >> I observed this by measuring the amount of memory consumed in each > > >> superstep. I create all the vectors of the superstep distance 0, this > > >> superstep has a jump in memory usage. The Problem is that the use not > > >> stabilizes, memory consumption always increases. I think it's > messages, > > >> because in eclipse I run on a only computer I can run a graph with > > 10,000 > > >> vertices in a cluster with 15 machines can not run a graph of 40 000 > > >> vertices. > > >> > > >> > > >> 2013/2/20 Thomas Jungblut <[email protected]> > > >> > > >> > > > > >> > > From what I could see all the messages that are sent remain in > > >> > > memory even if the superstep taking over. > > >> > > > > >> > > > >> > How did you measure this? Can you boil it down to a structure that > > holds > > >> > that messages? Have you profiled it? > > >> > > > >> > > >> > > >> > > >> -- > > >> Francisco Sanches > > >> > > > > > > > > >
