Re: Errors while running large graph

*giraph.zkJavaOpts* On 5/27/14, 10:27 AM, Praveen kumar s.k wrote: Do need to put this in the zookeeper configuration file or giraph job configuration? On Tue, May 27, 2014 at 12:14 PM, Avery Ching wrote: You might also want to check the zookeeper memory options. Some of our production jobs

Re: Errors while running large graph

Do need to put this in the zookeeper configuration file or giraph job configuration? On Tue, May 27, 2014 at 12:14 PM, Avery Ching wrote: > You might also want to check the zookeeper memory options. > > Some of our production jobs use parameters such as > > -Xmx5g -XX:ParallelGCThreads=4 -XX:+Use

Re: Errors while running large graph

You might also want to check the zookeeper memory options. Some of our production jobs use parameters such as -Xmx5g -XX:ParallelGCThreads=4 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxGCPauseMillis=100 Since the master doesn't use much memory letting zk have more is

Errors while running large graph

Hi All, I am getting several errors consistently while processing large graph. The code works when the size of the graph is in terms of GB's. we have implemented compression and removing the dead end nodes in de Bruijn graph My cluster settings are Cores WorkersRAM/Core Graphsize