Re: Setting Max Counters

2015-10-22 Thread Arjun Sharma
query-error-too-many-counters.html> > > On Wed, Oct 14, 2015 at 1:56 PM, Thomas Karampelas <tkar...@di.uoa.gr> > wrote: > >> Hi, >> >> Did you set mapreduce.job.counters.limit on all the machines of your >> cluster? >> >> Thomas >> >

Giraph Edges

2015-10-22 Thread Arjun Sharma
Hi, Is it "a must" to add Giraph edges? I have a way to keep track of the graph structure in my application, and can send messages to neighbor vertices by looking them up from the application. Is there anything to lose by not explicitly letting Giraph know about those edges (so basically from

Number of workers vs number of threads

2015-07-13 Thread Arjun Sharma
Hi, Many of the discussions on this forum suggest using one worker per physical machine, and increasing the number of threads per worker, versus using multiple workers per physical machine, with a less number of threads. This does not seem to be the case with my experiments. The cluster I am

Re: Optimal configuration for Giraph on YARN

2015-04-29 Thread Arjun Sharma
arguments) All of these were required to be manually set to get Giraph to run without having memory issues. Best regards, Steve On Thu, Apr 23, 2015 at 8:15 PM, Arjun Sharma as469...@gmail.com wrote: Just bumping up this thread, as I am having the same question as Steven's. Steven, did you

Re: Optimal configuration for Giraph on YARN

2015-04-23 Thread Arjun Sharma
Just bumping up this thread, as I am having the same question as Steven's. Steven, did you get to know if setting both mapreduce.map.cpu.vcores and yarn.nodemanager.resource.cpu-vcores is required? What happens if they are not set, while giraph.numComputeThreads is set? Are there any other

Custom assignment of partitions to workers

2015-03-25 Thread Arjun Sharma
Hi, I understand we can override the GraphPartitionerFactory class in order to achieve custom partitioning of vertices over partitions. Is there a way to do the same to enable assigning partitions to workers in a custom way (e.g., partition n should be assigned to worker m)? The reason is that it

Re: Giraph Partitioning

2015-02-25 Thread Arjun Sharma
not familiar with the out-of-core functionality 4) I'm not sure I'm basing this on the version of Giraph from this summer, not the most recent release, but I don't think this part has changed. May want to verify by looking at the code. Best, Matthew On Wed, Feb 25, 2015 at 3:25 AM, Arjun Sharma