Re: Optimal configuration for Giraph on YARN

2015-04-29 Thread Steven Harenberg
I would guess the same, but I don't know for sure. --Steve On Wed, Apr 29, 2015 at 12:21 PM, Arjun Sharma wrote: > Hi Steven, > > Thank you so much for your detailed reply! Actually, my second question > was about if we do not set mapreduce.map.cpu.vcores (defaults to 1) or > yarn.nodemanager.r

Re: Optimal configuration for Giraph on YARN

2015-04-29 Thread Arjun Sharma
Hi Steven, Thank you so much for your detailed reply! Actually, my second question was about if we do not set mapreduce.map.cpu.vcores (defaults to 1) or yarn.nodemanager.resource.cpu-vcores (defaults to 8), while we set giraph.numComputeThreads (say to 16). I expect every worker will run 16 threa

Re: Optimal configuration for Giraph on YARN

2015-04-29 Thread Steven Harenberg
Hey Arjun, I am glad someone finally responded to this thread. I am surprised no one else is trying to figure out these configuration settings... Here is my understanding of your questions (though I am not sure they are right): *Is setting both mapreduce.map.cpu.vcores and yarn.nodemanager.reso

Re: Optimal configuration for Giraph on YARN

2015-04-23 Thread Arjun Sharma
Just bumping up this thread, as I am having the same question as Steven's. Steven, did you get to know if setting both mapreduce.map.cpu.vcores and yarn.nodemanager.resource.cpu-vcores is required? What happens if they are not set, while giraph.numComputeThreads is set? Are there any other paramet

Optimal configuration for Giraph on YARN

2015-03-18 Thread Steven Harenberg
Hi all, Previously with MapReduceV1, the suggestion was to have a 1:1 correspondence between workers and compute nodes (machines) and set the number of the threads to be the number of cores per machines. To achieve this configuration, we would set "mapred.tasktracker.map.tasks.maximum=1". Since wo