Hi Dionysis, i was getting those error's because of the configuration issues (even after reading all the input values correctly it was not able to start compute for superstep 0).. it was because i was running program in wrong way..
(you can check till where your program is running by putting some print st's and checking o/p in syslog) following cmd worked for me hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar org.apache.giraph.examples.bc_random org.apache.giraph.examples.bc_randomVertex -_GMInputFormat ADJ org.apache.giraph.examples.bc_randomVertexInputFormat -_GMOutputFormat ADJ org.apache.giraph.examples.bc_randomVertexOutputFormat --K 5 -i inputbet -o out -w 1 @Claudio : Sorry actually my prog was exceeding counter limit which got resolved by increasing the the counter limit in mapred (gave 1024 i kw its never gona hit even close to that :-)) as you suggested, But its was just for small sample input (more of like linear graph), i am worried about actual graph which is huge and i might not even know the longest shortest path in graph (randomly changing) so.. was just asking is there a way i can cancel checking counter limit check.. On Fri, Feb 8, 2013 at 2:51 AM, Claudio Martella <claudio.marte...@gmail.com > wrote: > Sorry pradeep, I'm not sure I understand your problem. What about the 361 > supersteps and the counters limit change? > Did you solve it? > > > On Thu, Feb 7, 2013 at 5:43 PM, pradeep kumar <pradeep0...@gmail.com>wrote: > >> Hi Claudio >> actually i did increased limit on my laptop to 1024 (pseudo mode) it >> took 361 supersteps for 15 nodes and 18 edges (depth 10+), we will soon be >> testing this on cluster of 5 or 6 hope everything goes well. Is there a way >> i can disable this check..? i remember someone posting this earlier..!! but >> not sure how..;-( >> >> @Jan, yes, actually even i want to see results,..:-) >> >> PS thanks for all help.. >> >> >> >> >> On Thu, Feb 7, 2013 at 9:47 PM, Jan van der Lugt <janl...@gmail.com>wrote: >> >>> I never actually ran bc_random, so I don't know how many supersteps it >>> needs for different input sizes. If you could try this out, it would be >>> worthwhile to know. Especially the relation between input size / number of >>> supersteps. It's a limitation of the Pregel paradigm that you need quite a >>> few superstep to do BFS, not a lot we can do about this... >>> >>> >>> On Thu, Feb 7, 2013 at 3:47 PM, Claudio Martella < >>> claudio.marte...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> you have to increase the counters limit in your cluster. You can >>>> increase the limit by setting the property mapreduce.job.counters.limit in >>>> your mapred-site.xml accordingly. >>>> >>>> >>>> On Thu, Feb 7, 2013 at 4:34 PM, pradeep kumar <pradeep0...@gmail.com>wrote: >>>> >>>>> Hi Jan, >>>>> >>>>> well actually i just corrected my mistake.. i was running it wrong >>>>> way, thanks.. >>>>> just 1 more question on map-reduce counters >>>>> >>>>> giraph by default limits the superstep counters to 120 >>>>> but in bc_random master superstep itself exceeds this, so i had to >>>>> increase the limit..this was just for 20 nodes sample i/p i was trying >>>>> >>>>> my real data will be close to million nodes, so how can i manage for >>>>> such large input.. >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Feb 7, 2013 at 4:02 PM, pradeep kumar >>>>> <pradeep0...@gmail.com>wrote: >>>>> >>>>>> Hi lugt, >>>>>> >>>>>> Thanks for all your help.. program compiled successfully into jar.. >>>>>> but actually i am not able to run programs.. >>>>>> i tried following cmd to run >>>>>> >>>>>> 1) using GiraphRunner >>>>>> hadoop jar >>>>>> giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar >>>>>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.bc_randomVertex >>>>>> -vif org.apache.giraph.examples.bc_randomVertexInputFormat -vip inputbet >>>>>> -of org.apache.giraph.examples.bc_randomVertexOutputFormat -op outputbet >>>>>> -w 1 >>>>>> >>>>>> it does maps every thing successfully but then gives following error >>>>>> in log >>>>>> >>>>>> java.lang.IllegalStateException: run: Caught an unrecoverable exception >>>>>> waitFor: ExecutionException occurred while waiting for >>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@40fd6cd8 >>>>>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) >>>>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) >>>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) >>>>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:266) >>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>> at javax.security.auth.Subject.doAs(Subject.java:415) >>>>>> >>>>>> followed by some other errors >>>>>> >>>>>> 2) bc_random >>>>>> >>>>>> i actually couldnt understand all arg types here >>>>>> >>>>>> used : >>>>>> >>>>>> hadoop jar >>>>>> giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar >>>>>> org.apache.giraph.examples.bc_random >>>>>> org.apache.giraph.examples.bc_randomVertex --GMInputFormat ADJ >>>>>> org.apache.giraph.examples.bc_randomVertexInputFormat -i inputbet >>>>>> --GMOutputFormat ADJ >>>>>> org.apache.giraph.examples.bc_randomVertexOutputFormat -o outputbet -w 1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> but gives error for k procedure arg (not sure what to give as value >>>>>> here, even about about ip format i mentioned above ) >>>>>> >>>>>> >>>>>> I have uploaded sample input in hdfs in following format >>>>>> >>>>>> 10 4500 9 800 11 1000 12 1000 >>>>>> 11 5500 12 1100 11 800 >>>>>> >>>>>> nid val <eid val> >>>>>> >>>>>> did i made a mistake anywhere..? >>>>>> >>>>>> any suggestion will be a great help..!! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Feb 6, 2013 at 5:03 PM, Jan van der Lugt >>>>>> <janl...@gmail.com>wrote: >>>>>> >>>>>>> Hi Pradeep, >>>>>>> >>>>>>> You can check out the Green-Marl compiler from my Github ( >>>>>>> https://github.com/janlugt/Green-Marl), that one should be >>>>>>> compatible with the latest version of Giraph. Please let me know if >>>>>>> anything doesn't work for you. The changes will probably also be >>>>>>> accepted >>>>>>> upstream later this day. >>>>>>> >>>>>>> - Jan >>>>>>> >>>>>>> >>>>>>> On Wed, Feb 6, 2013 at 9:48 AM, Jan van der Lugt >>>>>>> <janl...@gmail.com>wrote: >>>>>>> >>>>>>>> It should work with version 0.2, but let me check. Since the API is >>>>>>>> changing all the time at the moment, it's very different to track all >>>>>>>> those >>>>>>>> changes. Probably they are numerous instances of the same error. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Feb 6, 2013 at 9:26 AM, pradeep kumar < >>>>>>>> pradeep0...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Lugt, >>>>>>>>> >>>>>>>>> Thanks for suggestion, i just tried with Green-Marl's bc_random, >>>>>>>>> But because of so many changes in Giraph codebase, imports and >>>>>>>>> many other functions in bc_random >>>>>>>>> generate lots of errors almost 100+ errors.. I tried with Giraph >>>>>>>>> 1.0 and current version of 2.0, >>>>>>>>> most of errors i have resolved but still there are many errors.. >>>>>>>>> Any suggestion on this..? >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Pradeep Kumar >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Pradeep Kumar >>>>> >>>> >>>> >>>> >>>> -- >>>> Claudio Martella >>>> claudio.marte...@gmail.com >>>> >>> >>> >> >> >> -- >> Pradeep Kumar >> > > > > -- > Claudio Martella > claudio.marte...@gmail.com > -- Pradeep Kumar
