Hi Yorn, I figured this out and detailed the solution in my post earlier this morning (6/23 2:46). The key is the following: -ca mapred.job.tracker=localhost:5431. Without this, you'll see the exception you detailed above.
--John On Thu, Jun 5, 2014 at 5:36 AM, Yørn de Jong <y...@uninett.no> wrote: > Hi group > > I have set up Giraph on a YARN cluster. I have no trouble running the > shortest paths example as described in [1], but when I try to run my own > algorithm, the program stops with: > > Exception in thread "main" Java.lang.IllegalArgumentException: > checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only > one worker since only 1 task at a time! > > When I change the command to -w 1, it stops with > > Exception in thread "main" java.lang.IllegalArgumentException: > checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run > in split master / worker mode since there is only 1 task at a time! > > The command I try to run is > > hadoop \ > jar > giraph-rank-1.1.0-SNAPSHOT-for-hadoop-2.3.0-cdh5.0.1-jar-with-dependencies.jar > \ > org.apache.giraph.GiraphRunner > no.uninett.yorn.giraph.computation.DOSRank \ > -eif no.uninett.yorn.giraph.format.io.NetflowCSVEdgeInputFormat \ > -eip /user/hdfs/trd_gw1_12_01_normalized.csv \ > -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat \ > -op /user/yarn/output \ > -wc org.apache.giraph.worker.DefaultWorkerContext \ > -w 5 \ > -yj > giraph-rank-1.1.0-SNAPSHOT-for-hadoop-2.3.0-cdh5.0.1-jar-with-dependencies.jar > > Where, naturally, no.uninett.yorn.giraph.computation.DOSRank is my own > algorithm, which is contained in > giraph-rank-1.1.0-SNAPSHOT-for-hadoop-2.3.0-cdh5.0.1-jar-with-dependencies.jar. > > giraph-rank is built using the same command I used to build the > giraph-examples project, and the pom.xml for giraph-rank is made by copying > from giraph-examples, and replacing «examples» with «rank». The command > used to build both projects is: > > mvn -Phadoop_yarn -Dhadoop.version=2.3.0-cdh5.0.1 clean package > -DskipTests > > Interestingly enough, when I change the input format to something else, I > get error messages having to do with type mismatches. This seems to suggest > that my EdgeInputFormat does start, but that the problem occurs while it > runs or after it runs. I don’t know how to debug this. > > Am I missing something? How can I get my algorithm to run? > The source code is available on [2]. > > [1] https://giraph.apache.org/quick_start.html#qs_section_5 > [2] https://scm.uninett.no/yorn/giraph >