Hello everybody I managed to compile Giraph 1.1 for Hadoop 2.6.0 with pure_yarn maven profile. I am running on a VM environment so I took the necessary actions as listed in the Giraph Quick Start Guide.
I am running on a functional single node Hadoop installation and I am trying to execute the example Shortest Path calculation, following the directions of the Giraph Quick Start Guide. The job is accepted and ran, but it ultimately fails (the job is listed as completed but the final state is failed). There's something wrong with the containers I think. Logs and other infos are just below the signature. Thanks in advance for any hint :) Alessio Here is the command I give: ############################# hadoop jar ~/Giraph_builds/giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip hdfs://VirtualMINT:9000/users/hadoop/input/giraph/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op hdfs://VirtualMINT:9000/users/hadoop/output/giraph/shortestpaths -yj /lib/giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar -w 2 ############################# Here is the output when the running fails, just below the classpath scanning output: #################################### 14/12/08 18:17:21 INFO impl.YarnClientImpl: Submitted application application_1418057611186_0006 14/12/08 18:17:21 INFO yarn.GiraphYarnClient: Got new appId after submission :application_1418057611186_0006 14/12/08 18:17:21 INFO yarn.GiraphYarnClient: GiraphApplicationMaster container request was submitted to ResourceManager for job: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation 14/12/08 18:17:22 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0.85 secs 14/12/08 18:17:22 INFO yarn.GiraphYarnClient: appattempt_1418057611186_0006_000001, State: ACCEPTED, Containers used: 1 14/12/08 18:17:26 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 4.88 secs 14/12/08 18:17:26 INFO yarn.GiraphYarnClient: appattempt_1418057611186_0006_000001, State: RUNNING, Containers used: 1 14/12/08 18:17:30 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 8.90 secs 14/12/08 18:17:30 INFO yarn.GiraphYarnClient: appattempt_1418057611186_0006_000001, State: RUNNING, Containers used: 1 14/12/08 18:17:31 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed cache directory for Giraph job. 14/12/08 18:17:31 INFO yarn.GiraphYarnClient: Completed Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total running time: 0 minutes, 9 seconds. Here is the "interesting part" of the log: 4/12/08 18:17:30 INFO yarn.GiraphApplicationMaster: Got response from RM for container ask, completedCnt=3 14/12/08 18:17:30 INFO yarn.GiraphApplicationMaster: Got container status for containerID=container_1418057611186_0006_01_000002, state=COMPLETE, exitStatus=127, diagnostics=Exception from container-launch. Container id: container_1418057611186_0006_01_000002 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 127 14/12/08 18:17:30 INFO yarn.GiraphApplicationMaster: Got container status for containerID=container_1418057611186_0006_01_000003, state=COMPLETE, exitStatus=127, diagnostics=Exception from container-launch. Container id: container_1418057611186_0006_01_000003 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 127 14/12/08 18:17:30 INFO yarn.GiraphApplicationMaster: Got container status for containerID=container_1418057611186_0006_01_000004, state=COMPLETE, exitStatus=127, diagnostics=Exception from container-launch. Container id: container_1418057611186_0006_01_000004 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 127 14/12/08 18:17:30 INFO yarn.GiraphApplicationMaster: All container compeleted. done = true 14/12/08 18:17:31 INFO yarn.GiraphApplicationMaster: Done true 14/12/08 18:17:31 INFO yarn.GiraphApplicationMaster: Forcefully terminating executors with done =:true 14/12/08 18:17:31 INFO yarn.GiraphApplicationMaster: Application completed. Stopping running containers 14/12/08 18:17:31 INFO impl.ContainerManagementProtocolProxy: Opening proxy : VirtualMINT:32971 14/12/08 18:17:31 INFO impl.ContainerManagementProtocolProxy: Opening proxy : VirtualMINT:32971 14/12/08 18:17:31 INFO impl.ContainerManagementProtocolProxy: Opening proxy : VirtualMINT:32971 14/12/08 18:17:31 INFO yarn.GiraphApplicationMaster: Application completed. Signalling finish to RM 14/12/08 18:17:31 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered. 14/12/08 18:17:31 INFO yarn.GiraphApplicationMaster: Giraph Application Master failed. exi ################################################
