Re: Scalability issue on Giraph

2015-12-14 Thread Hassan Eslami
Depending on the paper, some consider and some don't. Although, a fair measurement would consider the IO time as well. I just asked you not to consider the IO time, so to diagnose the problem easier, and isolate the time and only consider actual core computation and memory accesses. As a side note

Re: Scalability issue on Giraph

2015-12-14 Thread Karos Lotfifar
Thanks Hasan, My question is: is it the way others report superstep times in research papers such as Hama, Giraph, GraphLab, etc. Are they not considering the I/O time? (As they report good scalable reports for Pagerank calculation). The other case is that, even taking into account the computatio

Re: Scalability issue on Giraph

2015-12-14 Thread Hassan Eslami
Hi, Here is my take on it: It may be a good idea to isolate the time spent in compute supersteps. In order to do that, you can look at per-superstep timing metrics and aggregate the times for all supersteps except input (-1) and output (last) superstep. This eliminates the time for IO and turns yo

Scalability issue on Giraph

2015-12-14 Thread Foad Lotfifar
Hi, I have a scalability issue for Giraph and I can not find out where is the problem. --- Cluster specs: # nodes 1 # threads 32 Processor Intel Xeon 2.0GHz OSubuntu 32bit RAM 64GB --- Giraph specs HadoopApache Hado