Hi Tripti, finally I was able to run the test with success. It was an issue of permission since I was running as ale not as yarn. Let me say that now I’m able to run graph examples on Yarn 2.5.1. This is the final result:
14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total running time: 0 minutes, 21 seconds. Many thanks for your support, Alessandro Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh <tri...@yahoo-inc.com> ha scritto: > I don't know if u have access to this node. But if u do, u could check if the > file is indeed there and u have access to it. > > Sent from my iPhone > > On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <alenegr...@yahoo.it> wrote: > >> You are right it works, but now I receive the following error: >> >> SLF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in >> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in >> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from >> SCDynamicStore >> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: Could >> not configure the containerlaunch context for GiraphYarnTasks. >> at >> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:744) >> Caused by: java.io.FileNotFoundException: File does not exist: >> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0009/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> at >> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >> at >> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >> at >> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >> at >> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >> at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >> ... 6 more >> >> >> That justify the other error I receive in the task: >>>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >> >> >> Thanks, >> >> Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <tri...@yahoo-inc.com> >> ha scritto: >> >>> Why r u adding two jars? Example jar ideally contains core library so >>> everything should be available with just one example jar included >>> >>> Sent from my iPhone >>> >>> On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <alenegr...@yahoo.it> wrote: >>> >>>> Hi, >>>> now it seems better, I need to add: >>>> >>>> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>> >>>> >>>> Now it seems that after a lot of cycle it fail with this error: >>>> >>>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >>>> >>>> But in this case the error appear in task-3-stderr.log not in >>>> gam-stderr.log where there is the following error: >>>> >>>> LF4J: Class path contains multiple SLF4J bindings. >>>> SLF4J: Found binding in >>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>> SLF4J: Found binding in >>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>> SLF4J: Found binding in >>>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>>> explanation. >>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>>> 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from >>>> SCDynamicStore >>>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >>>> Could not configure the containerlaunch context for GiraphYarnTasks. >>>> at >>>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >>>> at >>>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >>>> at >>>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >>>> at >>>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>> at java.lang.Thread.run(Thread.java:744) >>>> Caused by: java.io.FileNotFoundException: File does not exist: >>>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0006/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>> at >>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >>>> at >>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >>>> at >>>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>>> at >>>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >>>> at >>>> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >>>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >>>> at >>>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >>>> ... 6 more >>>> >>>> >>>> Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh <tri...@yahoo-inc.com> >>>> ha scritto: >>>> >>>>> Hi, >>>>> I remember seeing this thread, but since i was already deep into fixing >>>>> this issue in production, I didn't really wait for the patch, not even >>>>> the munge symbol as I was not even able to compile the yarn profile( >>>>> sorry for being so ruthless to giraph :-( ) >>>>> I did later cross check with the patch and it was quite similar to my >>>>> change but sure it wouldn't work for all profiles(in fact some profiles >>>>> may even break due to Hadoop class incompatibility ) I can share those >>>>> changes with u once I get back to my computer but I don't think that >>>>> could have caused this problem. >>>>> Just try the small changed I mentioned in my previous reply and let me >>>>> know if that helped. >>>>> >>>>> Thanks, >>>>> Tripti >>>>> Sent from my iPhone >>>>> >>>>> On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <alenegr...@yahoo.it> >>>>> wrote: >>>>> >>>>>> Hi Tripti, >>>>>> let me add one info, As you can see from this thread of email: >>>>>> >>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%3cc3f75a98-bde8-4205-971f-1f7241e6e...@yahoo.it%3E >>>>>> >>>>>> I had some issue while compiling for Hadoop 2.5.1 could be that into the >>>>>> process I loose something? >>>>>> >>>>>> Thanks again, >>>>>> Alessandro >>>>>> >>>>>> >>>>>> >>>>>> Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro >>>>>> <alenegr...@yahoo.it> ha scritto: >>>>>> >>>>>>> Hi Tripti, >>>>>>> yes I run in this way: >>>>>>> >>>>>>> >>>>>>> $HADOOP_INSTALL/bin/hadoop jar >>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>> org.apache.giraph.GiraphRunner >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>> -vip /tiny_graph.txt -vof >>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>> /shortestpaths -w 1 -yj >>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>> >>>>>>> And this is the complete trace: >>>>>>> >>>>>>> 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from >>>>>>> SCDynamicStore >>>>>>> 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load >>>>>>> native-hadoop library for your platform... using builtin-java classes >>>>>>> where applicable >>>>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format >>>>>>> specified. Ensure your InputFormat does not require one. >>>>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format >>>>>>> specified. Ensure your OutputFormat does not require one. >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is: >>>>>>> hdfs://hadoop-master:9000/shortestpaths >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client >>>>>>> 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at >>>>>>> /0.0.0.0:8032 >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM >>>>>>> for, nodeId=10.61.137.28:59677, nodeAddress 10.61.137.28:8042, >>>>>>> nodeRackName /default-rack, nodeNumContainers 0 >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap >>>>>>> size to 1024MB. >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application >>>>>>> ID: application_1415264041937_0005 >>>>>>> 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id is >>>>>>> deprecated. Instead, use mapreduce.job.id >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for >>>>>>> the application master >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap .... >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in >>>>>>> LocalResources :: >>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415264041937_0005/giraph-conf.xml >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name . >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking . >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /contrib/capacity-scheduler >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>> container is populated. >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to >>>>>>> ASM >>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>> application_1415264041937_0005 >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>> submission :application_1415264041937_0005 >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >>>>>>> container request was submitted to ResourceManager for job: Giraph: >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>> 0,83 secs >>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: >>>>>>> appattempt_1415264041937_0005_000001, State: ACCEPTED, Containers used: >>>>>>> 1 >>>>>>> 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>> FAILED state, diagnostics show: Application >>>>>>> application_1415264041937_0005 failed 2 times due to AM Container for >>>>>>> appattempt_1415264041937_0005_000002 exited with exitCode: 1 due to: >>>>>>> Exception from container-launch: ExitCodeException exitCode=1: >>>>>>> ExitCodeException exitCode=1: >>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>> at >>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>> at >>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>> at >>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>> at >>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>> >>>>>>> >>>>>>> Container exited with a non-zero exit code 1 >>>>>>> .Failing this attempt.. Failing the application. >>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>> distributed cache directory for Giraph job. >>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, >>>>>>> total running time: 0 minutes, 3 seconds. >>>>>>> ale:~/extprj/giraph/myGiraph />yarn logs -applicationId >>>>>>> application_1415264041937_0005 >>>>>>> >>>>>>> As you can see there is this piece: >>>>>>> >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>> >>>>>>> >>>>>>> But I continue to receive the error: >>>>>>> >>>>>>> Thanks, >>>>>>> Alessandro >>>>>>> >>>>>>> >>>>>>> Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh >>>>>>> <tri...@yahoo-inc.com> ha scritto: >>>>>>> >>>>>>>> For GiraphApplicationMaster not found, the simple thing that worked >>>>>>>> for me was adding a -yj parameter to my command. >>>>>>>> (hadoop jar {jar-file-path} {path to giraph runner} >>>>>>>> {computation-class} {bunch of parameters} -yj {jar-file-name}) >>>>>>>> If u look at job logs, >>>>>>>> Do u see a log statement defining "LIB JARS"? >>>>>>>> >>>>>>>> Sent from my iPhone >>>>>>>> >>>>>>>> On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <alenegr...@yahoo.it> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> yea my logs shows that error. I follow this example: >>>>>>>>> >>>>>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3c53f4c689.5060...@web.de%3E >>>>>>>>> >>>>>>>>> without success. >>>>>>>>> Then I try to add all jars I can from giraph adding the following >>>>>>>>> lines into the hadoop-env.sh: >>>>>>>>> >>>>>>>>> export >>>>>>>>> MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>>> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB >>>>>>>>> >>>>>>>>> >>>>>>>>> when I run the application now I see this: >>>>>>>>> >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>>>> >>>>>>>>> >>>>>>>>> But again the same error. What I think is that there is no problem in >>>>>>>>> the library but while is starting up. >>>>>>>>> I saw this piece in the logs: >>>>>>>>> >>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>> container is populated. >>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application >>>>>>>>> to ASM >>>>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>>>> application_1415264041937_0005 >>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>>> submission :application_1415264041937_0005 >>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >>>>>>>>> container request was submitted to ResourceManager for job: Giraph: >>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>>>> 0,83 secs >>>>>>>>> >>>>>>>>> >>>>>>>>> That show that it is able to load GiraphApplicationMaster but fail >>>>>>>>> during the startup. >>>>>>>>> Where can I find more logs? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Alessandro >>>>>>>>> >>>>>>>>> >>>>>>>>> Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh >>>>>>>>> <tri...@yahoo-inc.com> ha scritto: >>>>>>>>> >>>>>>>>>> Can u run the following command to find container logs and post it >>>>>>>>>> here: >>>>>>>>>> >>>>>>>>>> yarn logs -applicationId application_1415203231170_0002 >>>>>>>>>> >>>>>>>>>> If u find this error: >>>>>>>>>> Could not find or load main class >>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>> It may just be due to non-availablity of the yarn jars >>>>>>>>>> Thanks, >>>>>>>>>> Tripti >>>>>>>>>> Tech Yahoo >>>>>>>>>> >>>>>>>>>> From: Jing Fan <fanjin...@gmail.com> >>>>>>>>>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >>>>>>>>>> Date: Thursday, November 6, 2014 at 1:56 AM >>>>>>>>>> To: "user@giraph.apache.org" <user@giraph.apache.org> >>>>>>>>>> Subject: Re: Help with Giraph on Yarn >>>>>>>>>> >>>>>>>>>> Then do you know to solve this error? >>>>>>>>>> >>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>>>> FAILED state, diagnostics show: Application >>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM Container >>>>>>>>>> for appattempt_1415203231170_0002_000002 exited with exitCode: 1 >>>>>>>>>> due to: Exception from container-launch. >>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>> Exit code: 1 >>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro >>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> in the mean time I found the reason why it hangs for long time here: >>>>>>>>>>> >>>>>>>>>>> base path checking . >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> It scan all the directory tree starting from current directory, if >>>>>>>>>>> you run it into your home (and you have a lot of files and >>>>>>>>>>> directories) it takes ages to complete. >>>>>>>>>>> >>>>>>>>>>> I continue to got the same error. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Alessandro >>>>>>>>>>> >>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan >>>>>>>>>>> <fanjin...@gmail.com> ha scritto: >>>>>>>>>>> >>>>>>>>>>>> I run with: >>>>>>>>>>>> >>>>>>>>>>>> hadoop jar >>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>> -vip /user/jing/tiny_graph.txt -vof >>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>> /user/jing/shortestpaths -w 1 -yj >>>>>>>>>>>> giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro >>>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>>> This is my complete logs with the command I run: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar >>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>> -vip /tiny_graph.txt -vof >>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>> /shortestpaths -w 1 >>>>>>>>>>>>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm info >>>>>>>>>>>>> from SCDynamicStore >>>>>>>>>>>>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load >>>>>>>>>>>>> native-hadoop library for your platform... using builtin-java >>>>>>>>>>>>> classes where applicable >>>>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input >>>>>>>>>>>>> format specified. Ensure your InputFormat does not require one. >>>>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output >>>>>>>>>>>>> format specified. Ensure your OutputFormat does not require one. >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path >>>>>>>>>>>>> is: hdfs://hadoop-master:9000/shortestpaths >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client >>>>>>>>>>>>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to >>>>>>>>>>>>> ResourceManager at /0.0.0.0:8032 >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report >>>>>>>>>>>>> from ASM for, nodeId=192.168.0.105:50658, nodeAddress >>>>>>>>>>>>> 192.168.0.105:8042, nodeRackName /default-rack, nodeNumContainers >>>>>>>>>>>>> 0 >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting per-task >>>>>>>>>>>>> heap size to 1024MB. >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new >>>>>>>>>>>>> Application ID: application_1415189069846_0004 >>>>>>>>>>>>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id >>>>>>>>>>>>> is deprecated. Instead, use mapreduce.job.id >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the environment >>>>>>>>>>>>> for the application master >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>>>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>> buildLocalResourceMap .... >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in >>>>>>>>>>>>> LocalResources :: >>>>>>>>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS : >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name . >>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>> /contrib/capacity-scheduler >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>>>>>> container is populated. >>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting >>>>>>>>>>>>> application to ASM >>>>>>>>>>>>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted application >>>>>>>>>>>>> application_1415189069846_0004 >>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>>>>>>> submission :application_1415189069846_0004 >>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>> GiraphApplicationMaster container request was submitted to >>>>>>>>>>>>> ResourceManager for job: Giraph: >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>> Elapsed: 0,82 secs >>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>> appattempt_1415189069846_0004_000001, State: ACCEPTED, Containers >>>>>>>>>>>>> used: 1 >>>>>>>>>>>>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>>>>>>> FAILED state, diagnostics show: Application >>>>>>>>>>>>> application_1415189069846_0004 failed 2 times due to AM Container >>>>>>>>>>>>> for appattempt_1415189069846_0004_000002 exited with exitCode: 1 >>>>>>>>>>>>> due to: Exception from container-launch: ExitCodeException >>>>>>>>>>>>> exitCode=1: >>>>>>>>>>>>> ExitCodeException exitCode=1: >>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>> at >>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>> at >>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>>>>>>>> at >>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>>>>>>>> at >>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>> at >>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>> at >>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Container exited with a non-zero exit code 1 >>>>>>>>>>>>> .Failing this attempt.. Failing the application. >>>>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>>>>>>>> distributed cache directory for Giraph job. >>>>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: >>>>>>>>>>>>> FAILED, total running time: 0 minutes, 3 seconds. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh >>>>>>>>>>>>> <tri...@yahoo-inc.com> ha scritto: >>>>>>>>>>>>> >>>>>>>>>>>>>> I faced the similar problem where the workflow just hangs. >>>>>>>>>>>>>> Can u see in ur logs if there is a task with id 0. >>>>>>>>>>>>>> I made a local fix and I could give u the patch if u see a >>>>>>>>>>>>>> similar problem and ur workflow is unable to find the zookeeper >>>>>>>>>>>>>> manager file and repeats the following message: >>>>>>>>>>>>>> For task xyz, got file 'null' >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Tripti >>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" >>>>>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> have you seen the hadoop logs? >>>>>>>>>>>>>>> Could you write here how are you running the command? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan >>>>>>>>>>>>>>> <fanjin...@gmail.com> ha scritto: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yeah, I add -yj option. The result is the same. Still error >>>>>>>>>>>>>>>> here. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro >>>>>>>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>>>>>>> Hi Jing, >>>>>>>>>>>>>>>>> I’m facing the same problem. Do you check the hadoop logs? In >>>>>>>>>>>>>>>>> my case I got: >>>>>>>>>>>>>>>>> Could not find or load main class >>>>>>>>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>>>>>>>>> Moreover I don’t know why my application hang for a long time >>>>>>>>>>>>>>>>> here: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> And then continue failing with your error into the output and >>>>>>>>>>>>>>>>> the other into the hadoop logs. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Did you test adding this? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -yj >>>>>>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan >>>>>>>>>>>>>>>>> <fanjin...@gmail.com> ha scritto: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I install the Giraph with Yarn. >>>>>>>>>>>>>>>>>> But when I run the program with following command: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> hadoop jar >>>>>>>>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>>>> -vif >>>>>>>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>>>>>>> -vip /user/j/tiny_graph.txt -vof >>>>>>>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>>>>>>> /user/j/shortestpaths -w 1 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The program exited with error: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>> Elapsed: 0.83 secs >>>>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>> Elapsed: 4.84 secs >>>>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>> Elapsed: 8.85 secs >>>>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>> Elapsed: 12.86 secs >>>>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>>>> reports FAILED state, diagnostics show: Application >>>>>>>>>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM >>>>>>>>>>>>>>>>>> Container for appattempt_1415203231170_0002_000002 exited >>>>>>>>>>>>>>>>>> with exitCode: 1 due to: Exception from container-launch. >>>>>>>>>>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>>>>>>>>>> Exit code: 1 >>>>>>>>>>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Anyone can help? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >>
