Hi Das, if you are referring to the following error: >> Caused by: java.io.FileNotFoundException: File does not exist: >> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0009/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >
I just run the script as yarn user, since as you can see from the error, it try to access to the graph cache on the yarn user. What error are you receiving? Regards, Alessandro Il giorno 17/nov/2014, alle ore 17:10, D Adams <dadam...@gmail.com> ha scritto: > Greetings, > Alessandro, I keep getting the same errors you were getting but from > reading the thread, I still cannot figure out how to remedy it. Could you > please break down how you solved this issue? Thank you > > V/r > Das > > On Nov 16, 2014 3:40 PM, "Eli Reisman" <apache.mail...@gmail.com> wrote: > Sounds like you got it figured out since last time I checked this list, sorry > it was a pain. Feel free to drop a JIRA ticket if we can make help message > etc. for -yj option more clear, there's lots to improve there. > > On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro <alenegr...@yahoo.it> wrote: > Hi Tripti, > finally I was able to run the test with success. It was an issue of > permission since I was running as ale not as yarn. > Let me say that now I’m able to run graph examples on Yarn 2.5.1. This is the > final result: > > 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total > running time: 0 minutes, 21 seconds. > > > Many thanks for your support, > Alessandro > > Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh <tri...@yahoo-inc.com> ha > scritto: > >> I don't know if u have access to this node. But if u do, u could check if >> the file is indeed there and u have access to it. >> >> Sent from my iPhone >> >> On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <alenegr...@yahoo.it> wrote: >> >>> You are right it works, but now I receive the following error: >>> >>> SLF4J: Class path contains multiple SLF4J bindings. >>> SLF4J: Found binding in >>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>> SLF4J: Found binding in >>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>> explanation. >>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>> 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from >>> SCDynamicStore >>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >>> Could not configure the containerlaunch context for GiraphYarnTasks. >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: java.io.FileNotFoundException: File does not exist: >>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0009/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >>> at >>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >>> at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >>> ... 6 more >>> >>> >>> That justify the other error I receive in the task: >>>>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >>> >>> >>> Thanks, >>> >>> Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <tri...@yahoo-inc.com> >>> ha scritto: >>> >>>> Why r u adding two jars? Example jar ideally contains core library so >>>> everything should be available with just one example jar included >>>> >>>> Sent from my iPhone >>>> >>>> On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <alenegr...@yahoo.it> wrote: >>>> >>>>> Hi, >>>>> now it seems better, I need to add: >>>>> >>>>> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> >>>>> >>>>> Now it seems that after a lot of cycle it fail with this error: >>>>> >>>>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >>>>> >>>>> But in this case the error appear in task-3-stderr.log not in >>>>> gam-stderr.log where there is the following error: >>>>> >>>>> LF4J: Class path contains multiple SLF4J bindings. >>>>> SLF4J: Found binding in >>>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> SLF4J: Found binding in >>>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> SLF4J: Found binding in >>>>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>>>> explanation. >>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>>>> 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from >>>>> SCDynamicStore >>>>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >>>>> Could not configure the containerlaunch context for GiraphYarnTasks. >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> Caused by: java.io.FileNotFoundException: File does not exist: >>>>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0006/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >>>>> at >>>>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >>>>> at >>>>> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >>>>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >>>>> ... 6 more >>>>> >>>>> >>>>> Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh >>>>> <tri...@yahoo-inc.com> ha scritto: >>>>> >>>>>> Hi, >>>>>> I remember seeing this thread, but since i was already deep into fixing >>>>>> this issue in production, I didn't really wait for the patch, not even >>>>>> the munge symbol as I was not even able to compile the yarn profile( >>>>>> sorry for being so ruthless to giraph :-( ) >>>>>> I did later cross check with the patch and it was quite similar to my >>>>>> change but sure it wouldn't work for all profiles(in fact some profiles >>>>>> may even break due to Hadoop class incompatibility ) I can share those >>>>>> changes with u once I get back to my computer but I don't think that >>>>>> could have caused this problem. >>>>>> Just try the small changed I mentioned in my previous reply and let me >>>>>> know if that helped. >>>>>> >>>>>> Thanks, >>>>>> Tripti >>>>>> Sent from my iPhone >>>>>> >>>>>> On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <alenegr...@yahoo.it> >>>>>> wrote: >>>>>> >>>>>>> Hi Tripti, >>>>>>> let me add one info, As you can see from this thread of email: >>>>>>> >>>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%3cc3f75a98-bde8-4205-971f-1f7241e6e...@yahoo.it%3E >>>>>>> >>>>>>> I had some issue while compiling for Hadoop 2.5.1 could be that into >>>>>>> the process I loose something? >>>>>>> >>>>>>> Thanks again, >>>>>>> Alessandro >>>>>>> >>>>>>> >>>>>>> >>>>>>> Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro >>>>>>> <alenegr...@yahoo.it> ha scritto: >>>>>>> >>>>>>>> Hi Tripti, >>>>>>>> yes I run in this way: >>>>>>>> >>>>>>>> >>>>>>>> $HADOOP_INSTALL/bin/hadoop jar >>>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>> -vip /tiny_graph.txt -vof >>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>> /shortestpaths -w 1 -yj >>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> >>>>>>>> And this is the complete trace: >>>>>>>> >>>>>>>> 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from >>>>>>>> SCDynamicStore >>>>>>>> 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load >>>>>>>> native-hadoop library for your platform... using builtin-java classes >>>>>>>> where applicable >>>>>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format >>>>>>>> specified. Ensure your InputFormat does not require one. >>>>>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format >>>>>>>> specified. Ensure your OutputFormat does not require one. >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is: >>>>>>>> hdfs://hadoop-master:9000/shortestpaths >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client >>>>>>>> 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager >>>>>>>> at /0.0.0.0:8032 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM >>>>>>>> for, nodeId=10.61.137.28:59677, nodeAddress 10.61.137.28:8042, >>>>>>>> nodeRackName /default-rack, nodeNumContainers 0 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap >>>>>>>> size to 1024MB. >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application >>>>>>>> ID: application_1415264041937_0005 >>>>>>>> 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id is >>>>>>>> deprecated. Instead, use mapreduce.job.id >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for >>>>>>>> the application master >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap >>>>>>>> .... >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in >>>>>>>> LocalResources :: >>>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415264041937_0005/giraph-conf.xml >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name . >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking . >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /contrib/capacity-scheduler >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>> container is populated. >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application >>>>>>>> to ASM >>>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>>> application_1415264041937_0005 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>> submission :application_1415264041937_0005 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >>>>>>>> container request was submitted to ResourceManager for job: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>>> 0,83 secs >>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: >>>>>>>> appattempt_1415264041937_0005_000001, State: ACCEPTED, Containers >>>>>>>> used: 1 >>>>>>>> 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>> FAILED state, diagnostics show: Application >>>>>>>> application_1415264041937_0005 failed 2 times due to AM Container for >>>>>>>> appattempt_1415264041937_0005_000002 exited with exitCode: 1 due to: >>>>>>>> Exception from container-launch: ExitCodeException exitCode=1: >>>>>>>> ExitCodeException exitCode=1: >>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>> at >>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>>> >>>>>>>> >>>>>>>> Container exited with a non-zero exit code 1 >>>>>>>> .Failing this attempt.. Failing the application. >>>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>>> distributed cache directory for Giraph job. >>>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, >>>>>>>> total running time: 0 minutes, 3 seconds. >>>>>>>> ale:~/extprj/giraph/myGiraph />yarn logs -applicationId >>>>>>>> application_1415264041937_0005 >>>>>>>> >>>>>>>> As you can see there is this piece: >>>>>>>> >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> >>>>>>>> >>>>>>>> But I continue to receive the error: >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Alessandro >>>>>>>> >>>>>>>> >>>>>>>> Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh >>>>>>>> <tri...@yahoo-inc.com> ha scritto: >>>>>>>> >>>>>>>>> For GiraphApplicationMaster not found, the simple thing that worked >>>>>>>>> for me was adding a -yj parameter to my command. >>>>>>>>> (hadoop jar {jar-file-path} {path to giraph runner} >>>>>>>>> {computation-class} {bunch of parameters} -yj {jar-file-name}) >>>>>>>>> If u look at job logs, >>>>>>>>> Do u see a log statement defining "LIB JARS"? >>>>>>>>> >>>>>>>>> Sent from my iPhone >>>>>>>>> >>>>>>>>> On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <alenegr...@yahoo.it> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> yea my logs shows that error. I follow this example: >>>>>>>>>> >>>>>>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3c53f4c689.5060...@web.de%3E >>>>>>>>>> >>>>>>>>>> without success. >>>>>>>>>> Then I try to add all jars I can from giraph adding the following >>>>>>>>>> lines into the hadoop-env.sh: >>>>>>>>>> >>>>>>>>>> export >>>>>>>>>> MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>>>> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> when I run the application now I see this: >>>>>>>>>> >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> But again the same error. What I think is that there is no problem >>>>>>>>>> in the library but while is starting up. >>>>>>>>>> I saw this piece in the logs: >>>>>>>>>> >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>>> container is populated. >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application >>>>>>>>>> to ASM >>>>>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>>>>> application_1415264041937_0005 >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>>>> submission :application_1415264041937_0005 >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>>>> GiraphApplicationMaster container request was submitted to >>>>>>>>>> ResourceManager for job: Giraph: >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>>>>> 0,83 secs >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> That show that it is able to load GiraphApplicationMaster but fail >>>>>>>>>> during the startup. >>>>>>>>>> Where can I find more logs? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Alessandro >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh >>>>>>>>>> <tri...@yahoo-inc.com> ha scritto: >>>>>>>>>> >>>>>>>>>>> Can u run the following command to find container logs and post it >>>>>>>>>>> here: >>>>>>>>>>> >>>>>>>>>>> yarn logs -applicationId application_1415203231170_0002 >>>>>>>>>>> >>>>>>>>>>> If u find this error: >>>>>>>>>>> Could not find or load main class >>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>>> It may just be due to non-availablity of the yarn jars >>>>>>>>>>> Thanks, >>>>>>>>>>> Tripti >>>>>>>>>>> Tech Yahoo >>>>>>>>>>> >>>>>>>>>>> From: Jing Fan <fanjin...@gmail.com> >>>>>>>>>>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >>>>>>>>>>> Date: Thursday, November 6, 2014 at 1:56 AM >>>>>>>>>>> To: "user@giraph.apache.org" <user@giraph.apache.org> >>>>>>>>>>> Subject: Re: Help with Giraph on Yarn >>>>>>>>>>> >>>>>>>>>>> Then do you know to solve this error? >>>>>>>>>>> >>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>>>>> FAILED state, diagnostics show: Application >>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM Container >>>>>>>>>>> for appattempt_1415203231170_0002_000002 exited with exitCode: 1 >>>>>>>>>>> due to: Exception from container-launch. >>>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>>> Exit code: 1 >>>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro >>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> in the mean time I found the reason why it hangs for long time >>>>>>>>>>>> here: >>>>>>>>>>>> >>>>>>>>>>>> base path checking . >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> It scan all the directory tree starting from current directory, if >>>>>>>>>>>> you run it into your home (and you have a lot of files and >>>>>>>>>>>> directories) it takes ages to complete. >>>>>>>>>>>> >>>>>>>>>>>> I continue to got the same error. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Alessandro >>>>>>>>>>>> >>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan >>>>>>>>>>>> <fanjin...@gmail.com> ha scritto: >>>>>>>>>>>> >>>>>>>>>>>>> I run with: >>>>>>>>>>>>> >>>>>>>>>>>>> hadoop jar >>>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>> -vip /user/jing/tiny_graph.txt -vof >>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>> /user/jing/shortestpaths -w 1 -yj >>>>>>>>>>>>> giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro >>>>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>>>> This is my complete logs with the command I run: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar >>>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>>> -vip /tiny_graph.txt -vof >>>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>>> /shortestpaths -w 1 >>>>>>>>>>>>>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm >>>>>>>>>>>>>> info from SCDynamicStore >>>>>>>>>>>>>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load >>>>>>>>>>>>>> native-hadoop library for your platform... using builtin-java >>>>>>>>>>>>>> classes where applicable >>>>>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input >>>>>>>>>>>>>> format specified. Ensure your InputFormat does not require one. >>>>>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output >>>>>>>>>>>>>> format specified. Ensure your OutputFormat does not require one. >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path >>>>>>>>>>>>>> is: hdfs://hadoop-master:9000/shortestpaths >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to >>>>>>>>>>>>>> ResourceManager at /0.0.0.0:8032 >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report >>>>>>>>>>>>>> from ASM for, nodeId=192.168.0.105:50658, nodeAddress >>>>>>>>>>>>>> 192.168.0.105:8042, nodeRackName /default-rack, >>>>>>>>>>>>>> nodeNumContainers 0 >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting >>>>>>>>>>>>>> per-task heap size to 1024MB. >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new >>>>>>>>>>>>>> Application ID: application_1415189069846_0004 >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id >>>>>>>>>>>>>> is deprecated. Instead, use mapreduce.job.id >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the >>>>>>>>>>>>>> environment for the application master >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>>>>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> buildLocalResourceMap .... >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in >>>>>>>>>>>>>> LocalResources :: >>>>>>>>>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS : >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name . >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /contrib/capacity-scheduler >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>>>>>>> container is populated. >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting >>>>>>>>>>>>>> application to ASM >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted >>>>>>>>>>>>>> application application_1415189069846_0004 >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId >>>>>>>>>>>>>> after submission :application_1415189069846_0004 >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> GiraphApplicationMaster container request was submitted to >>>>>>>>>>>>>> ResourceManager for job: Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>> Elapsed: 0,82 secs >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> appattempt_1415189069846_0004_000001, State: ACCEPTED, >>>>>>>>>>>>>> Containers used: 1 >>>>>>>>>>>>>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>> reports FAILED state, diagnostics show: Application >>>>>>>>>>>>>> application_1415189069846_0004 failed 2 times due to AM >>>>>>>>>>>>>> Container for appattempt_1415189069846_0004_000002 exited with >>>>>>>>>>>>>> exitCode: 1 due to: Exception from container-launch: >>>>>>>>>>>>>> ExitCodeException exitCode=1: >>>>>>>>>>>>>> ExitCodeException exitCode=1: >>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Container exited with a non-zero exit code 1 >>>>>>>>>>>>>> .Failing this attempt.. Failing the application. >>>>>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>>>>>>>>> distributed cache directory for Giraph job. >>>>>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: >>>>>>>>>>>>>> FAILED, total running time: 0 minutes, 3 seconds. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh >>>>>>>>>>>>>> <tri...@yahoo-inc.com> ha scritto: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I faced the similar problem where the workflow just hangs. >>>>>>>>>>>>>>> Can u see in ur logs if there is a task with id 0. >>>>>>>>>>>>>>> I made a local fix and I could give u the patch if u see a >>>>>>>>>>>>>>> similar problem and ur workflow is unable to find the zookeeper >>>>>>>>>>>>>>> manager file and repeats the following message: >>>>>>>>>>>>>>> For task xyz, got file 'null' >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Tripti >>>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" >>>>>>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> have you seen the hadoop logs? >>>>>>>>>>>>>>>> Could you write here how are you running the command? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan >>>>>>>>>>>>>>>> <fanjin...@gmail.com> ha scritto: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yeah, I add -yj option. The result is the same. Still error >>>>>>>>>>>>>>>>> here. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro >>>>>>>>>>>>>>>>> <alenegr...@yahoo.it> wrote: >>>>>>>>>>>>>>>>>> Hi Jing, >>>>>>>>>>>>>>>>>> I’m facing the same problem. Do you check the hadoop logs? >>>>>>>>>>>>>>>>>> In my case I got: >>>>>>>>>>>>>>>>>> Could not find or load main class >>>>>>>>>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>>>>>>>>>> Moreover I don’t know why my application hang for a long >>>>>>>>>>>>>>>>>> time here: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> And then continue failing with your error into the output >>>>>>>>>>>>>>>>>> and the other into the hadoop logs. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Did you test adding this? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -yj >>>>>>>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan >>>>>>>>>>>>>>>>>> <fanjin...@gmail.com> ha scritto: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I install the Giraph with Yarn. >>>>>>>>>>>>>>>>>>> But when I run the program with following command: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> hadoop jar >>>>>>>>>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>>>>> -vif >>>>>>>>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>>>>>>>> -vip /user/j/tiny_graph.txt -vof >>>>>>>>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat >>>>>>>>>>>>>>>>>>> -op /user/j/shortestpaths -w 1 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The program exited with error: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 0.83 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 4.84 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 8.85 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 12.86 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>>>>> reports FAILED state, diagnostics show: Application >>>>>>>>>>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM >>>>>>>>>>>>>>>>>>> Container for appattempt_1415203231170_0002_000002 exited >>>>>>>>>>>>>>>>>>> with exitCode: 1 due to: Exception from container-launch. >>>>>>>>>>>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>>>>>>>>>>> Exit code: 1 >>>>>>>>>>>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Anyone can help? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > >
