Hi Matthew, It is my understanding that to run LocalJobRunner, you need to change your Hadoop configuration a bit.
That is what I did: (1) in core-site.xml, comment out the fs.default.name property (2) in mapred-site.xml, comment out the mapred.job.tracker property Hope it helps. There are probably other smarter ways of doing this, and I would like to know as well. Wei From: Matthew Laird <lai...@sfu.ca> To: user@giraph.apache.org, Date: 10/13/2013 04:57 PM Subject: LocalJobRunner & examples I've been trying to work through the Quick Start examples and adapting it to my local installation, I'm running Hadoop 2.0.5-alpha across 4 machines. I've done the checkout from github sudo git clone https://github.com/apache/giraph.git Applied the patch to ZooKeeperManager http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff Fixed the catching of the exception in giraph-core/src/main/java/org/apache/giraph/job/HaltApplicationUtils.java Then building with mvn clean install -Phadoop_2.0.3 -DskipTests Then when I go to run the examples, I set -w 4 because I am running a split master setup. Unfortunately this leads to: lairdm@x007:/opt/giraph$ $HADOOP_HOME/bin/hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.0.3-alpha-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /in/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths -w 4 13/10/13 13:50:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 13/10/13 13:50:48 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 13/10/13 13:50:48 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one. 13/10/13 13:50:49 WARN conf.Configuration: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max 13/10/13 13:50:49 WARN conf.Configuration: mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb 13/10/13 13:50:49 WARN conf.Configuration: mapred.job.reduce.memory.mb is deprecated. Instead, use mapreduce.reduce.memory.mb 13/10/13 13:50:49 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 13/10/13 13:50:49 WARN conf.Configuration: mapreduce.user.classpath.first is deprecated. Instead, use mapreduce.job.user.classpath.first 13/10/13 13:50:49 WARN conf.Configuration: mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts 13/10/13 13:50:49 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4) 13/10/13 13:50:49 WARN conf.Configuration: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only one worker since only 1 task at a time! at org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration (GiraphJob.java:159) at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:233) at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) I'm not sure what parameters I need to give giraph to signal not to use LocalJobRunner. Does anyone have a suggestion? Thanks.
<<inline: graycol.gif>>
