Hi Das, I mean as user you run Hadoop yarn resource manager. Alessandro
> Il giorno 20/nov/2014, alle ore 19:22, D Adams <dadam...@gmail.com> ha > scritto: > > Alessandro, > I'm not sure what you mean, should I create a new user on my system > named 'yarn'? I'm new to both hadoop and giraph, so I'm not sure. > > Tripti, > I'll get those logs as soon as I can. > > Thank you both, > > V/r > Das > >> On Nov 20, 2014 3:13 AM, "Alessandro Negro" <alenegr...@yahoo.it> wrote: >> Hi Tripti, >> I agree that a more detailed error log could be useful. >> >> Thanks, >> Alessandro >> >> >> >> >> Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh <tri...@yahoo-inc.com> >> ha scritto: >> >> > --_000_D092AB9C1CB42triptiyahooinccom_ >> > Content-Type: text/plain; charset="Windows-1252" >> > Content-Transfer-Encoding: quoted-printable >> > >> > Hi Alessandro, >> > >> > I guess u r talking about the documentation on Giraph webpage which >> > mention= >> > s addition of this new option which is more or less mandatory for yarn >> > base= >> > d profiles. >> > When I first ran the Giraph on yarn without the proper =96yj option, there >> > = >> > was no way I could figure out from the Application/Container logs that >> > this= >> > was the issue. >> > I think, it=92ll be a good idea to have this message in the logs for easy >> > d= >> > ebugging. >> > >> > Thanks, >> > Tripti. >> > >> > >> > From: Alessandro Negro <alenegr...@yahoo.it<mailto:alenegr...@yahoo.it>> >> > Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" >> > <user@gir= >> > aph.apache.org<mailto:user@giraph.apache.org>> >> > Date: Tuesday, November 18, 2014 at 4:31 PM >> > To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" >> > <u...@giraph.ap= >> > ache.org<mailto:user@giraph.apache.org>> >> > Subject: Re: Help with Giraph on Yarn >> > >> > Hi Eli, >> > no I think that the help message is clear enough, generally when I read >> > =93= >> > jar=94 I always mean absolute path for jar, but this is a my personal >> > misun= >> > derstanding. >> > >> > Thanks, >> > Alessandro >> > >> > >> > Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman >> > <apache.mail...@gmail.co= >> > m<mailto:apache.mail...@gmail.com>> ha scritto: >> > >> > Sounds like you got it figured out since last time I checked this list, >> > sor= >> > ry it was a pain. Feel free to drop a JIRA ticket if we can make help >> > messa= >> > ge etc. for -yj option more clear, there's lots to improve there. >> > >> > On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro >> > <alenegr...@yahoo.it<mailt= >> > o:alenegr...@yahoo.it>> wrote: >> > Hi Tripti, >> > finally I was able to run the test with success. It was an issue of >> > permiss= >> > ion since I was running as ale not as yarn. >> > Let me say that now I=92m able to run graph examples on Yarn 2.5.1. This >> > is= >> > the final result: >> > >> > 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: >> > org.apache.= >> > giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total running >> > ti= >> > me: 0 minutes, 21 seconds. >> > >> > >> > Many thanks for your support, >> > Alessandro >> > >> > Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh >> > <tri...@yahoo-inc.com<m= >> > ailto:tri...@yahoo-inc.com>> ha scritto: >> > >> > I don't know if u have access to this node. But if u do, u could check if >> > t= >> > he file is indeed there and u have access to it. >> > >> > Sent from my iPhone >> > >> > On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" >> > <alenegr...@yahoo.it<mailto:= >> > alenegr...@yahoo.it>> wrote: >> > >> > You are right it works, but now I receive the following error: >> > >> > SLF4J: Class path contains multiple SLF4J bindings. >> > SLF4J: Found binding in >> > [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use= >> > rcache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-exam= >> > ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/i= >> > mpl/StaticLoggerBinder.class] >> > SLF4J: Found binding in >> > [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo= >> > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> > explana= >> > tion. >> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> > 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from >> > SCD= >> > ynamicStore >> > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >> > Coul= >> > d not configure the containerlaunch context for GiraphYarnTasks. >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph= >> > ApplicationMaster.java:391) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat= >> > ionMaster.java:78) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b= >> > uildContainerLaunchContext(GiraphApplicationMaster.java:522) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r= >> > un(GiraphApplicationMaster.java:479) >> > at >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= >> > a:1145) >> > at >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= >> > va:615) >> > at java.lang.Thread.run(Thread.java:744) >> > Caused by: java.io.FileNotFoundException: File does not exist: >> > hdfs://hadoo= >> > p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_000= >> > 9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> > at >> > org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy= >> > stem.java:1072) >> > at >> > org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy= >> > stem.java:1064) >> > at >> > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv= >> > er.java:81) >> > at >> > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi= >> > leSystem.java:1064) >> > at >> > org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153= >> > ) >> > at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph= >> > ApplicationMaster.java:387) >> > ... 6 more >> > >> > >> > That justify the other error I receive in the task: >> > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >> > >> > Thanks, >> > >> > Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh >> > <tri...@yahoo-inc.com<m= >> > ailto:tri...@yahoo-inc.com>> ha scritto: >> > >> > Why r u adding two jars? Example jar ideally contains core library so >> > every= >> > thing should be available with just one example jar included >> > >> > Sent from my iPhone >> > >> > On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" >> > <alenegr...@yahoo.it<mailto:= >> > alenegr...@yahoo.it>> wrote: >> > >> > Hi, >> > now it seems better, I need to add: >> > >> > giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-exa= >> > mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> > >> > >> > Now it seems that after a lot of cycle it fail with this error: >> > >> > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >> > >> > But in this case the error appear in task-3-stderr.log not in >> > gam-stderr.l= >> > og where there is the following error: >> > >> > LF4J: Class path contains multiple SLF4J bindings. >> > SLF4J: Found binding in >> > [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use= >> > rcache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.= >> > 0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/Stati= >> > cLoggerBinder.class] >> > SLF4J: Found binding in >> > [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use= >> > rcache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-exam= >> > ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/i= >> > mpl/StaticLoggerBinder.class] >> > SLF4J: Found binding in >> > [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo= >> > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> > explana= >> > tion. >> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> > 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from >> > SCDy= >> > namicStore >> > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >> > Coul= >> > d not configure the containerlaunch context for GiraphYarnTasks. >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph= >> > ApplicationMaster.java:391) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat= >> > ionMaster.java:78) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b= >> > uildContainerLaunchContext(GiraphApplicationMaster.java:522) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r= >> > un(GiraphApplicationMaster.java:479) >> > at >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= >> > a:1145) >> > at >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= >> > va:615) >> > at java.lang.Thread.run(Thread.java:744) >> > Caused by: java.io.FileNotFoundException: File does not exist: >> > hdfs://hadoo= >> > p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_000= >> > 6/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> > at >> > org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy= >> > stem.java:1072) >> > at >> > org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy= >> > stem.java:1064) >> > at >> > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv= >> > er.java:81) >> > at >> > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi= >> > leSystem.java:1064) >> > at >> > org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153= >> > ) >> > at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >> > at >> > org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph= >> > ApplicationMaster.java:387) >> > ... 6 more >> > >> > >> > Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh >> > <tri...@yahoo-inc.com<m= >> > ailto:tri...@yahoo-inc.com>> ha scritto: >> > >> > Hi, >> > I remember seeing this thread, but since i was already deep into fixing >> > thi= >> > s issue in production, I didn't really wait for the patch, not even the >> > mun= >> > ge symbol as I was not even able to compile the yarn profile( sorry for >> > bei= >> > ng so ruthless to giraph :-( ) >> > I did later cross check with the patch and it was quite similar to my >> > chang= >> > e but sure it wouldn't work for all profiles(in fact some profiles may >> > even= >> > break due to Hadoop class incompatibility ) I can share those changes with= >> > u once I get back to my computer but I don't think that could have caused = >> > this problem. >> > Just try the small changed I mentioned in my previous reply and let me >> > know= >> > if that helped. >> > >> > Thanks, >> > Tripti >> > Sent from my iPhone >> > >> > On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" >> > <alenegr...@yahoo.it<mailto:= >> > alenegr...@yahoo.it>> wrote: >> > >> > Hi Tripti, >> > let me add one info, As you can see from this thread of email: >> > >> > http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%3CC3F75A9= >> > 8-bde8-4205-971f-1f7241e6e...@yahoo.it%3E >> > >> > I had some issue while compiling for Hadoop 2.5.1 could be that into the >> > pr= >> > ocess I loose something? >> > >> > Thanks again, >> > Alessandro >> > >> > >> > >> > Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro >> > <alenegro81@yahoo.i= >> > t<mailto:alenegr...@yahoo.it>> ha scritto: >> > >> > Hi Tripti, >> > yes I run in this way: >> > >> > >> > $HADOOP_INSTALL/bin/hadoop jar >> > /Users/ale/extprj/giraph/giraph-examples/tar= >> > get/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.j= >> > ar org.apache.giraph.GiraphRunner >> > org.apache.giraph.examples.SimpleShortest= >> > PathsComputation -vif >> > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubl= >> > eVertexInputFormat -vip /tiny_graph.txt -vof >> > org.apache.giraph.io.formats.I= >> > dWithValueTextOutputFormat -op /shortestpaths -w 1 -yj >> > /Users/ale/extprj/gi= >> > raph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dep= >> > endencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-exampl= >> > es-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> > >> > And this is the complete trace: >> > >> > 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from >> > SCDy= >> > namicStore >> > 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load native-hadoop >> > = >> > library for your platform... using builtin-java classes where applicable >> > 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format >> > speci= >> > fied. Ensure your InputFormat does not require one. >> > 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format >> > spec= >> > ified. Ensure your OutputFormat does not require one. >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is: >> > hdfs://= >> > hadoop-master:9000/shortestpaths >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client >> > 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at >> > /0.= >> > 0.0.0:8032<http://0.0.0.0:8032/> >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM >> > for,= >> > nodeId=3D10.61.137.28:59677<http://10.61.137.28:59677/>, nodeAddress 10.61= >> > .137.28:8042<http://10.61.137.28:8042/>, nodeRackName /default-rack, >> > nodeNu= >> > mContainers 0 >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap >> > size= >> > to 1024MB. >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application ID: >> > = >> > application_1415264041937_0005 >> > 14/11/06 10:06:06 INFO Configuration.deprecation: >> > mapred.job.id<http://mapr= >> > ed.job.id/> is deprecated. Instead, use >> > mapreduce.job.id<http://mapreduce.j= >> > ob.id/> >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for the >> > a= >> > pplication master >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM >> > :{CLASSPAT= >> > H=3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/comm= >> > on/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/= >> > hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/s= >> > hare/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED= >> > _HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/l= >> > ib/*} >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap .... >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in LocalResources >> > ::= >> > hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_14152= >> > 64041937_0005/giraph-conf.xml >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >> > :/Users/ale/extprj/g= >> > iraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-de= >> > pendencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examp= >> > les-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name . >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking . >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/etc/hadoop >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/etc/hadoop >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/common/lib/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/common/lib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/common/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/common >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/hdfs >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/hdfs >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/hdfs/lib/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/hdfs/lib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/hdfs/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/hdfs >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/yarn/lib/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/yarn/lib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/yarn/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/yarn >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/mapreduce/lib/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/mapreduce/lib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/hadoop/mapreduce/* >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/mapreduce >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/myLib/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/myLib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /Users/ale/extprj/gi= >> > raph/myGiraph/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /Users/ale/extprj= >> > /giraph/myGiraph >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /Users/ale/extprj/gi= >> > raph/myGiraph/lib/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /Users/ale/extprj= >> > /giraph/myGiraph/lib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /contrib/capacity-sc= >> > heduler/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /contrib/capacity= >> > -scheduler >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext >> > = >> > for GiraphApplicationMaster launch container is populated. >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM >> > 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >> > applicati= >> > on_1415264041937_0005 >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >> > submissio= >> > n :application_1415264041937_0005 >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >> > conta= >> > iner request was submitted to ResourceManager for job: Giraph: >> > org.apache.g= >> > iraph.examples.SimpleShortestPathsComputation >> > 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >> > org.apache.giraph.exa= >> > mples.SimpleShortestPathsComputation, Elapsed: 0,83 secs >> > 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: >> > appattempt_1415264041937_0005= >> > _000001, State: ACCEPTED, Containers used: 1 >> > 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph: >> > org.apache.giraph.ex= >> > amples.SimpleShortestPathsComputation reports FAILED state, diagnostics >> > sho= >> > w: Application application_1415264041937_0005 failed 2 times due to AM >> > Cont= >> > ainer for appattempt_1415264041937_0005_000002 exited with exitCode: 1 >> > due= >> > to: Exception from container-launch: ExitCodeException exitCode=3D1: >> > ExitCodeException exitCode=3D1: >> > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >> > at org.apache.hadoop.util.Shell.run(Shell.java:455) >> > at >> > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702= >> > ) >> > at >> > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launc= >> > hContainer(DefaultContainerExecutor.java:195) >> > at >> > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Cont= >> > ainerLaunch.call(ContainerLaunch.java:300) >> > at >> > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Cont= >> > ainerLaunch.call(ContainerLaunch.java:81) >> > at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> > at >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= >> > a:1145) >> > at >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= >> > va:615) >> > at java.lang.Thread.run(Thread.java:744) >> > >> > >> > Container exited with a non-zero exit code 1 >> > .Failing this attempt.. Failing the application. >> > 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed >> > = >> > cache directory for Giraph job. >> > 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph: >> > org.apache.= >> > giraph.examples.SimpleShortestPathsComputation: FAILED, total running >> > time:= >> > 0 minutes, 3 seconds. >> > ale:~/extprj/giraph/myGiraph />yarn logs -applicationId >> > application_1415264= >> > 041937_0005 >> > >> > As you can see there is this piece: >> > >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >> > :/Users/ale/extprj/g= >> > iraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-de= >> > pendencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examp= >> > les-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> > >> > >> > But I continue to receive the error: >> > >> > Thanks, >> > Alessandro >> > >> > >> > Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh >> > <tri...@yahoo-inc.com<m= >> > ailto:tri...@yahoo-inc.com>> ha scritto: >> > >> > For GiraphApplicationMaster not found, the simple thing that worked for me >> > = >> > was adding a -yj parameter to my command. >> > (hadoop jar {jar-file-path} {path to giraph runner} {computation-class} >> > {bu= >> > nch of parameters} -yj {jar-file-name}) >> > If u look at job logs, >> > Do u see a log statement defining "LIB JARS"? >> > >> > Sent from my iPhone >> > >> > On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" >> > <alenegr...@yahoo.it<mailto:= >> > alenegr...@yahoo.it>> wrote: >> > >> > Hi, >> > yea my logs shows that error. I follow this example: >> > >> > http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C68= >> > 9.5060...@web.de%3E >> > >> > without success. >> > Then I try to add all jars I can from giraph adding the following lines >> > int= >> > o the hadoop-env.sh: >> > >> > export >> > MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/g= >> > iraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar >> > export HADOOP_CLASSPATH=3D$HADOOP_CLASSPATH:$MYLIB >> > >> > >> > when I run the application now I see this: >> > >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/hadoop/mapreduce >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /opt/yarn/hadoop-2.5= >> > .1/share/myLib/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /opt/yarn/hadoop-= >> > 2.5.1/share/myLib >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /Users/ale/extprj/gi= >> > raph/myGiraph/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /Users/ale/extprj= >> > /giraph/myGiraph >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >> > /Users/ale/extprj/gi= >> > raph/myGiraph/lib/*.jar >> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> > /Users/ale/extprj= >> > /giraph/myGiraph/lib >> > >> > >> > But again the same error. What I think is that there is no problem in the >> > l= >> > ibrary but while is starting up. >> > I saw this piece in the logs: >> > >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext >> > = >> > for GiraphApplicationMaster launch container is populated. >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM >> > 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >> > applicati= >> > on_1415264041937_0005 >> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >> > submissio= >> > n :application_141526 >>
