Re: Help with Giraph on Yarn

;> >>* >> >>* NOTE: ContainerId will serve as MR TaskID for Giraph tasks. >> >>* YARN container 1 is always AppMaster, so the least container id we >> will >> >>* ever get from YARN for a Giraph task is container id 2. Giraph on >&

Re: Help with Giraph on Yarn

Thanks, > Tripti > From: Tripti Singh > Reply-To: "user@giraph.apache.org" > Date: Sunday, November 23, 2014 at 12:23 PM > > To: "user@giraph.apache.org" > Subject: Re: Help with Giraph on Yarn > > Hi Das, > Could u try what Alessandro is suggesting.

Re: Help with Giraph on Yarn

raph.apache.org>> Date: Sunday, November 23, 2014 at 12:23 PM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Hi Das, Could u try what Alessandro is suggesting. Giraph-example jar has all

Re: Help with Giraph on Yarn

Negro mailto:alenegr...@yahoo.it>> Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Date: Sunday, November 23, 2014 at 5:10 AM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.

Re: Help with Giraph on Yarn

0)) - Registered file in >>> LocalResources :: >>> hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_1416310= >>> 496070_0001/giraph-conf.xml >>> 2014-11-18 05:36:37,694 INFO >>> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] >

Re: Help with Giraph on Yarn

.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #2] >> impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing >> Event EventType: QUERY_CONTAINER for Container >> container_1416310496070_0001_01_03 >> 2014-11-18 05:36:37,771 INFO >> [org.apache

Re: Help with Giraph on Yarn

> I agree that a more detailed error log could be useful. > > Thanks, > Alessandro > > > > > Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh > m> > > ha scritto: > > --_000_D092AB9C1CB42triptiyahooinccom_ > Content-Type: text/plain; charset=3D&qu

Re: Help with Giraph on Yarn

RY_CONTAINER for Container >> container_1416310496070_0001_01_02 >> >> >> >> On Thu, Nov 20, 2014 at 1:32 PM, Alessandro Negro >> wrote: >> >>> Hi Das, >>> I mean as user you run Hadoop yarn resource manager. >>>

Re: Help with Giraph on Yarn

s, >>> Alessandro >>> >>> >>> >>> >>> Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh m> >>> ha scritto: >>> >>>> --_000_D092AB9C1CB42triptiyahooinccom_ >>>> Content-Type: text/plain; cha

Re: Help with Giraph on Yarn

gt; Thanks, >> > Tripti. >> > >> > >> > From: Alessandro Negro mailto:alenegr...@yahoo.it >> >> >> > Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" >> > > aph.apache.org<mailto:user@giraph.apache.

Re: Help with Giraph on Yarn

gt;> > this= >> > was the issue. >> > I think, it=92ll be a good idea to have this message in the logs for easy >> > d= >> > ebugging. >> > >> > Thanks, >> > Tripti. >> > >> > >> > From: Alessandro Negro mailto:alenegr

Re: Help with Giraph on Yarn

good idea to have this message in the logs for > easy d= > > ebugging. > > > > Thanks, > > Tripti. > > > > > > From: Alessandro Negro mailto:alenegr...@yahoo.it>> > > Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>&

Re: Help with Giraph on Yarn

:alenegr...@yahoo.it>> > Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" aph.apache.org<mailto:user@giraph.apache.org>> > Date: Tuesday, November 18, 2014 at 4:31 PM > To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" ache.

Re: Help with Giraph on Yarn

iraph.apache.org" > Date: Tuesday, November 18, 2014 at 4:31 PM > > To: "user@giraph.apache.org" > Subject: Re: Help with Giraph on Yarn > > Hi Eli, > no I think that the help message is clear enough, generally when I read > “jar” I always mean absolute pa

Re: Help with Giraph on Yarn

user@giraph.apache.org>> Date: Tuesday, November 18, 2014 at 4:31 PM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Hi Eli, no I think that the help message is clear enough, generally when

Re: Help with Giraph on Yarn

) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>&

Re: Help with Giraph on Yarn

t;>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>> at >

Re: Help with Giraph on Yarn

myGiraph/lib/*.jar >> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB >> >> >> when I run the application now I see this: >> >> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >> 14/1

Re: Help with Giraph on Yarn

I saw this piece in the logs: > > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: > ApplicationSumbissionContext for GiraphApplicationMaster launch container > is populated. > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM > 14/11/06 10:06:06 INFO impl.Yar

Re: Help with Giraph on Yarn

xited with a non-zero exit code 1 >>>>>>> .Failing this attempt.. Failing the application. >>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>> distributed cache directory for Giraph job. >>>&

Re: Help with Giraph on Yarn

;> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB >> >> >> when I run the application now I see this: >> >> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >> 14/11/06 10:06:06 INFO yarn.YarnUt

Re: Help with Giraph on Yarn

apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Date: Thursday, November 6, 2014 at 1:56 AM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Then do yo

Re: Help with Giraph on Yarn

the library but while is starting up. > I saw this piece in the logs: > > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: > ApplicationSumbissionContext for GiraphApplicationMaster launch container > is populated. > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM > 14/11/06

Re: Help with Giraph on Yarn

raphApplicationMaster It may just be due to non-availablity of the yarn jars Thanks, Tripti Tech Yahoo From: Jing Fan mailto:fanjin...@gmail.com>> Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Date: Thursday, Nove

Re: Help with Giraph on Yarn

le-name}) >>>>>> If u look at job logs, >>>>>> Do u see a log statement defining "LIB JARS"? >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> On 06-Nov-2014, at 2:45

Re: Help with Giraph on Yarn

e.org>" mailto:user@giraph.apache.org>> Date: Thursday, November 6, 2014 at 1:56 AM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Then do you know to solve this error? 14/11/

Re: Help with Giraph on Yarn

t;>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /Users/ale/extprj/giraph/myGiraph >>>>> 14/11/06 10:0

Re: Help with Giraph on Yarn

M To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Then do you know to solve this error? 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputa

Re: Help with Giraph on Yarn

rceManager for job: Giraph: >>> org.apache.giraph.examples.SimpleShortestPathsComputation >>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,83 >>> secs >>&

Re: Help with Giraph on Yarn

Tech Yahoo From: Jing Fan mailto:fanjin...@gmail.com>> Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Date: Thursday, November 6, 2014 at 1:56 AM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>"

Re: Help with Giraph on Yarn

> >>> Can u run the following command to find container logs and post it here: >>> >>> yarn logs -applicationId application_1415203231170_0002 >>> >>> If u find this error: >>> Could not find or load main class >>> org.apache.gira

Re: Help with Giraph on Yarn

t;> Date: Thursday, November 6, 2014 at 1:56 AM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Then do you know to solve this error? 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Girap

Re: Help with Giraph on Yarn

t; Tech Yahoo > > From: Jing Fan > Reply-To: "user@giraph.apache.org" > Date: Thursday, November 6, 2014 at 1:56 AM > To: "user@giraph.apache.org" > Subject: Re: Help with Giraph on Yarn > > Then do you know to

Re: Help with Giraph on Yarn

ser@giraph.apache.org>" mailto:user@giraph.apache.org>> Subject: Re: Help with Giraph on Yarn Then do you know to solve this error? 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED state, diagnostics show: Application

Re: Help with Giraph on Yarn

Then do you know to solve this error? 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED state, diagnostics show: Application application_1415203231170_0002 failed 2 times due to AM Container for appattempt_1415203231170_

Re: Help with Giraph on Yarn

Hi, in the mean time I found the reason why it hangs for long time here: base path checking . It scan all the directory tree starting from current directory, if you run it into your home (and you have a lot of files and directories) it takes ages to complete. I continue to got the same error.

Re: Help with Giraph on Yarn

I run with: hadoop jar giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vi

Re: Help with Giraph on Yarn

This is my complete logs with the command I run: ale:~ />$HADOOP_INSTALL/bin/hadoop jar /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputa

Re: Help with Giraph on Yarn

I faced the similar problem where the workflow just hangs. Can u see in ur logs if there is a task with id 0. I made a local fix and I could give u the patch if u see a similar problem and ur workflow is unable to find the zookeeper manager file and repeats the following message: For task xyz, go

Re: Help with Giraph on Yarn

Hi, have you seen the hadoop logs? Could you write here how are you running the command? Thanks, Alessandro Il giorno 05/nov/2014, alle ore 18:04, Jing Fan ha scritto: > Yeah, I add -yj option. The result is the same. Still error here. > > On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro wr

Re: Help with Giraph on Yarn

Yeah, I add -yj option. The result is the same. Still error here. On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro wrote: > Hi Jing, > I’m facing the same problem. Do you check the hadoop logs? In my case I > got: > > Could not find or load main class > org.apache.giraph.yarn.GiraphApplication

Re: Help with Giraph on Yarn

Hi Jing, I’m facing the same problem. Do you check the hadoop logs? In my case I got: Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster Moreover I don’t know why my application hang for a long time here: 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking . A

Help with Giraph on Yarn

Hi, I install the Giraph with Yarn. But when I run the program with following command: hadoop jar giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vi