try to increase the mappers' heap space through mapred.child.java.opts.
On Mon, Oct 7, 2013 at 1:18 PM, Silvio Di gregorio < silvio.digrego...@gmail.com> wrote: > I think now my issue is another, thanks to all. > > > > java.lang.IllegalStateException: run: Caught an unrecoverable exception > waitFor: ExecutionException occurred while waiting for > org.apache.giraph.utils.ProgressableUtils$FutureWaitable@73a01e8a > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) > > > ...... > Caused by: java.lang.IllegalStateException: waitFor: ExecutionException > occurred while waiting for > org.apache.giraph.utils.ProgressableUtils$FutureWaitable@73a01e8a > at > org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:151) > > > ... 7 more > Caused by: java.util.concurrent.ExecutionException: > java.lang.OutOfMemoryError: *Java heap space* > at > org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:143) > > > ... 13 more > > > > 2013/10/7 Claudio Martella <claudio.marte...@gmail.com> > >> OK, thanks. >> >> I really have to push that patch in. >> >> >> On Mon, Oct 7, 2013 at 12:17 PM, Silvio Di gregorio < >> silvio.digrego...@gmail.com> wrote: >> >>> yes i do, >>> i have seen this in your post in: >>> http://www.mail-archive.com/user@giraph.apache.org/msg00957.html >>> >>> excuse me if i had checked in the mail-achive first I would have >>> avoided the last post. >>> >>> Now zk issue are resolved. >>> >>> >>> 2013/10/7 Claudio Martella <claudio.marte...@gmail.com> >>> >>>> Can you try applying this one first? >>>> >>>> http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff >>>> >>>> >>>> On Mon, Oct 7, 2013 at 8:40 AM, Silvio Di gregorio < >>>> silvio.digrego...@gmail.com> wrote: >>>> >>>>> >>>>> >>>>> >>>>> >>>>> *As i said i have builded* >>>>> >>>>> >>>>> >>>>> *giraph-examples-1.0.0-for-hadoop-2.0.0-cdh4.1.2-jar-with-dependencies.jar* >>>>> >>>>> >>>>> >>>>> *for cdh4, successfully. The job start to monitoring the success rate:* >>>>> >>>>> >>>>> >>>>> *13/10/07 08:28:45 INFO mapred.JobClient: map 0% reduce 0%* >>>>> >>>>> >>>>> >>>>> *but then* >>>>> >>>>> >>>>> >>>>> *Error running child >>>>> java.lang.IllegalStateException: run: Caught an unrecoverable exception >>>>> java.io.FileNotFoundException: File >>>>> _bsp/_defaultZkManagerDir/job_201309181636_0678/_zkServer does not exist. >>>>> * >>>>> >>>>> >>>>> >>>>> *...* >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: >>>>> File _bsp/_defaultZkManagerDir/job_201309181636_0678/_zkServer does not >>>>> exist.* >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> 2013/10/5 Silvio Di gregorio <silvio.digrego...@gmail.com> >>>>> >>>>>> I ha ve build w/ hadoop_cdh4.1.2 parameter. >>>>>> Something is changed, monday i report the result. Now the farm is >>>>>> closed. >>>>>> Il giorno 05/ott/2013 14:06, "Claudio Martella" < >>>>>> claudio.marte...@gmail.com> ha scritto: >>>>>> >>>>>> Oh, right, -vof is in trunk. Anyway it looks like you built giraph >>>>>>> for the wrong profile. You mentioned you're running on 2.0, but your >>>>>>> giraph >>>>>>> is built for 0.20.203. try building with a profile for your hadoop >>>>>>> version. >>>>>>> >>>>>>> >>>>>>> On Fri, Oct 4, 2013 at 2:35 PM, Silvio Di gregorio < >>>>>>> silvio.digrego...@gmail.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> org.apache.commons.cli.UnrecognizedOptionException: Unrecognized >>>>>>>> option: -vof >>>>>>>> >>>>>>>> in 1.0 version is >>>>>>>> -of,--outputFormat <arg> Vertex output format >>>>>>>> -op,--outputPath <arg> Vertex output path >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 2013/10/4 Claudio Martella <claudio.marte...@gmail.com> >>>>>>>> >>>>>>>>> did you try the argument (-vof) i suggested? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregorio < >>>>>>>>> silvio.digrego...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> i've specified -of >>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat >>>>>>>>>> >>>>>>>>>> but the same error was produced >>>>>>>>>> >>>>>>>>>> Exception in thread "main" >>>>>>>>>> java.lang.IncompatibleClassChangeError: Found interface >>>>>>>>>> org.apache.hadoop.mapreduce.JobContext, but class was expected >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:984) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945) >>>>>>>>>> at java.security.AccessController.doPrivileged(Native >>>>>>>>>> Method) >>>>>>>>>> at javax.security.auth.Subject.doAs(Subject.java:396) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945) >>>>>>>>>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:566) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596) >>>>>>>>>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:237) >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>>>>> Method) >>>>>>>>>> at >>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>>>>>>> at >>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2013/10/4 Claudio Martella <claudio.marte...@gmail.com> >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> you need to specify the vertex outputformat class (-vof option), >>>>>>>>>>> e.g. org.apache.giraph.io.formats.IdWithValueTextOutputFormat. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio < >>>>>>>>>>> silvio.digrego...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I hope I have sent to the right address. >>>>>>>>>>>> >>>>>>>>>>>> i have a graph (directed and unweighted) stored in hdfs like a >>>>>>>>>>>> adjacency list (140Milions of edges 6Milions of vertex) >>>>>>>>>>>> >>>>>>>>>>>> node<tab>neighbors >>>>>>>>>>>> >>>>>>>>>>>> 23 2 1 3 4 3 >>>>>>>>>>>> >>>>>>>>>>>> 1 999 99923 909 ....... >>>>>>>>>>>> >>>>>>>>>>>> ...... >>>>>>>>>>>> >>>>>>>>>>>> hadoop version Hadoop 2.0.0-cdh4.3.0 - java 1.6 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I have executed the giraph-1.0 connected components example, in >>>>>>>>>>>> this fashion >>>>>>>>>>>> >>>>>>>>>>>>> > hadoop jar /usr/local/giraph/giraph-examples/target/giraph- >>>>>>>>>>>>> examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>> org.apache.giraph.examples.ConnectedComponentsVertex >>>>>>>>>>>>> -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat >>>>>>>>>>>>> -vip /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> and then fail with: >>>>>>>>>>>> >>>>>>>>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No edge input >>>>>>>>>>>> format specified. Ensure your InputFormat does not require one. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No output >>>>>>>>>>>> format specified. Ensure your OutputFormat does not require one. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 13/10/04 09:28:30 INFO job.GiraphJob: run: Since checkpointing >>>>>>>>>>>> is disabled (default), do not allow any task retries (setting >>>>>>>>>>>> mapred.map.max.attempts = 0, old value = 4) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 13/10/04 09:28:31 WARN mapred.JobClient: Use >>>>>>>>>>>> GenericOptionsParser for parsing the arguments. Applications should >>>>>>>>>>>> implement Tool for the same. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 13/10/04 09:28:31 INFO mapred.JobClient: Cleaning up the >>>>>>>>>>>> staging area hdfs:// >>>>>>>>>>>> srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535 >>>>>>>>>>>> >>>>>>>>>>>> Exception in thread "main" >>>>>>>>>>>> java.lang.IncompatibleClassChangeError: Found interface >>>>>>>>>>>> org.apache.hadoop.mapreduce.JobContext, but class was expected >>>>>>>>>>>> >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >>>>>>>>>>>> >>>>>>>>>>>> ...... >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks in advance >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Claudio Martella >>>>>>>>>>> claudio.marte...@gmail.com >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Claudio Martella >>>>>>>>> claudio.marte...@gmail.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Claudio Martella >>>>>>> claudio.marte...@gmail.com >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Claudio Martella >>>> claudio.marte...@gmail.com >>>> >>> >>> >> >> >> -- >> Claudio Martella >> claudio.marte...@gmail.com >> > > -- Claudio Martella claudio.marte...@gmail.com
