I have successfully run the shortest path example using Avery's sample input
data. I am now attempting to run the shortest-path algorithm on a much larger
data set (300,000 nodes) and I am running into errors. I have a 4-node cluster
and am running the following command:
./giraph
Hi Bence,
on older version of hadoop there is a hard limit on counters, which a
job cannot modify. Since the counters are not crucial for the
functioning of giraph, you can turn them off by setting
giraph.useSuperstepCounters to false in your job config.
I would also recommend looking into the
Bence,
I set that value to 100 - I think there is a recommendation to set this
very high. Remember to reboot you cluster after making the change.
Jon
On Wed, Nov 28, 2012 at 6:07 AM, Magyar, Bence (US SSA)
bence.mag...@baesystems.com wrote:
I have successfully run the shortest path
Oh, forgot one thing. You need to set the number of partitions to use
single each thread works on a single partition at a time.
Try -Dhash.userPartitionCount=number of threads
On 11/28/12 5:29 AM, Alexandros Daglis wrote:
Dear Avery,
I followed your advice, but the application seems to be
Thank you Andre,
Setting giraph.useSuperstepCounters = false
solved my issue. The job still hung at 100% and then eventually completed
successfully.
-Bence
-Original Message-
From: André Kelpe [mailto:efeshundert...@googlemail.com]
Sent: Wednesday, November 28, 2012 10:45 AM
To:
Hi,
I am trying to run this workflow which uses Giraph.
I am able to succesfully run the Giraph job when I use lesser no. of mappers
and less data. But it fails for more mappers.
This is what the logs say for master and worker nodes:
Master Node:
2012-11-29 00:01:10,235 INFO [main]
Hi,
Update on this one.
I was able to resolve this error with the patch here:
https://issues.apache.org/jira/browse/GIRAPH-391
Thanks,
Tripti.
From: Yahoo! Inc. tri...@yahoo-inc.commailto:tri...@yahoo-inc.com
Reply-To: user@giraph.apache.orgmailto:user@giraph.apache.org
