Just to clarify. It is not due to pregel. The number of iterations of BFS is bound by the longest shortest path (the diameter). If you know the diameter and the number of random sources, you can compute the total number of supersteps and hence counters.
Hope this helps. Claudio On Thursday, February 7, 2013, Jan van der Lugt wrote: > I never actually ran bc_random, so I don't know how many supersteps it > needs for different input sizes. If you could try this out, it would be > worthwhile to know. Especially the relation between input size / number of > supersteps. It's a limitation of the Pregel paradigm that you need quite a > few superstep to do BFS, not a lot we can do about this... > > On Thu, Feb 7, 2013 at 3:47 PM, Claudio Martella < > claudio.marte...@gmail.com> wrote: > > Hi, > > you have to increase the counters limit in your cluster. You can increase > the limit by setting the property mapreduce.job.counters.limit in your > mapred-site.xml accordingly. > > > On Thu, Feb 7, 2013 at 4:34 PM, pradeep kumar <pradeep0...@gmail.com>wrote: > > Hi Jan, > > well actually i just corrected my mistake.. i was running it wrong way, > thanks.. > just 1 more question on map-reduce counters > > giraph by default limits the superstep counters to 120 > but in bc_random master superstep itself exceeds this, so i had to > increase the limit..this was just for 20 nodes sample i/p i was trying > > my real data will be close to million nodes, so how can i manage for such > large input.. > > > > > On Thu, Feb 7, 2013 at 4:02 PM, pradeep kumar <pradeep0...@gmail.com>wrote: > > Hi lugt, > > Thanks for all your help.. program compiled successfully into jar.. > but actually i am not able to run programs.. > i tried following cmd to run > > 1) using GiraphRunner > hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner org.apache.giraph.examples.bc_randomVertex > -vif org.apache.giraph.examples.bc_randomVertexInputFormat -vip inputbet > -of org.apache.giraph.examples.bc_randomVertexOutputFormat -op outputbet > -w 1 > > it does maps every thing successfully but then gives following error in log > > java.lang.IllegalStateException: run: Caught an unrecoverable exception > waitFor: ExecutionException occurred while waiting for > org.apache.giraph.utils.ProgressableUtils$FutureWaitable@40fd6cd8 > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) > at org.apache.hadoop.mapred.Child$4.run(Child.java:266) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > > followed by some other errors > > 2) bc_random > > i actually couldnt understand all arg types here > > used : > > hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar > org.apache.giraph.examples.bc_random > org.apache.giraph.examples.bc_randomVertex --GMInputFormat ADJ > org.apache.giraph.examples.bc_randomVertexInputFormat -i inputbet > --GMOutputFormat ADJ org.apache.giraph.examples.bc_randomVertexOutputFormat > -o outputbet -w 1 > > > > > but gives error for k procedure arg (not sure what to give as value here, > even about about ip format i mentioned above ) > > > I have uploaded sample input in hdfs in following format > > 10 4500 9 800 11 1000 12 1000 > 11 5500 12 1100 11 800 > > nid val <eid val> > > did i made a mistake anywhere..? > > any suggestion will be a great help..!! > > > > > > On Wed, Feb 6, 2013 at 5:03 PM, Jan van der Lugt <janl...@gmail.com>wrote: > > Hi Pradeep, > > You can check out the Green-Marl compiler from my Github ( > https://github.com/janlugt/Green-Marl), that one should be compatible > with t > > -- Claudio Martella claudio.marte...@gmail.com
