Re: Zookeeper Problems when running Giraph (PageRankVertex) on large graphs.

2014-09-24 Thread Lukas Nalezenec
Hi, I had similar problems. Files that Giraph wrote to zookeeper were over limit so zookeeper crushed. In my case i had too many partitions with too long input and checkpoint paths. You can try to get logs from the Zookeeper process. Lukas On 24.9.2014 14:31, Fontana, Peter C. wrote: Hello,

Zookeeper Problems when running Giraph (PageRankVertex) on large graphs.

2014-09-24 Thread Fontana, Peter C.
Hello, I am trying to run the PageRankVertex code on a large graph. I successfully got it to run on smaller examples, but when I try to run it on a large example (100M nodes, 10B edges, 300GB space), it does not finish. I get the following error. java.lang.IllegalStateException: run: Caught an

Re: understanding failing my job, Giraph/Hadoop memory usage, under-utilized nodes, and moving forward

2014-09-24 Thread Matthew Cornell
I cannot thank you enough, Matthew. You've given me a lot to experiment with. -- matt On Mon, Sep 22, 2014 at 2:10 PM, Matthew Saltz wrote: > Hi Matthew, > > I answered a few of your questions in-line (unfortunately they might not > help the larger problem, but hopefully it'll help a bit). > > Be