Re: workload used to measure Giraph performance number

I have used the Random Message Benchmark as a starting workload. On 10/2/2013 10:59 AM, Wei Zhang wrote: Hi, I am interested in measuring some performance numbers of Giraph on my machine. I am wondering are there some pointers where I can get some (configurable) reasonably large workload t

Re: Out of core execution has no effect on GC crash

Alexander: You might try turning off the GC Overhead limit (-XX:-UseGCOverheadLimit) Also you could turn on verbose GC logging (-verbose:gc -Xloggc:/tmp/@taskid@.gc) to see what is happening. Because the OOC still has to create and destroy objects I suspect that the heap is just getting r

Re: Optimal configuration for benchmark

Christian: I have actually been looking for a more general Giraph benchmark and would love to test/play with what you have. To answer your questions we need to first assume a dedicated cluster where your test is the only one running. For number of mappers we will assume th

Re: Running SimpleShortestPathsVertex with ToolRunner

Best, Jonathan On Thu, Apr 4, 2013 at 9:35 AM, David Boyd mailto:db...@data-tactics-corp.com>> wrote: All of the documentation for Giraph on both the Githiub wiki and the the Apache site is out of date as it related to the 0.2 version. Currently I have tested the followin

Re: Running SimpleShortestPathsVertex with ToolRunner

All of the documentation for Giraph on both the Githiub wiki and the the Apache site is out of date as it related to the 0.2 version. Currently I have tested the following benchmark examples from the core library: PageRankBenchmark ShortestPathsBenchmark AggregatorsBenchmark RandomMessageBenc

Re: Getting Giraph to build/run with CDH4 (cannot initialize cluster - check mapreduce.framework.name)

nning cluster, they like to instantiate their own local resources (ZK, Hadoop single-node) for the tests. If your example jobs run with "hadoop jar" on the cluster than thats what matters, you're all set. On Mon, Feb 25, 2013 at 4:03 PM, David Boyd mailto:db...@data-tact

Re: Getting Giraph to build/run with CDH4 (cannot initialize cluster - check mapreduce.framework.name)

va:350) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021) Below is the surefire report stack trace: On 2/25/2013 6:55 PM, Sandy Ryza wrote: Great to hear it helped. Are you able to provide the full stack trace for that exception? thanks, Sandy On Mon, Feb 25, 2013 at 3

Re: Getting Giraph to build/run with CDH4 (cannot initialize cluster - check mapreduce.framework.name)

nt to 2.0.0-mr1-cdh4.1.1. (https://ccp.cloudera.com/display/CDH4DOC/Managing+Hadoop+API+Dependencies+in+CDH4) Does that help? -Sandy On Mon, Feb 25, 2013 at 8:26 AM, David Boyd mailto:db...@data-tactics-corp.com>> wrote: All: I am trying to get the Giraph 0.2 snapshot (pulled

Getting Giraph to build/run with CDH4 (cannot initialize cluster - check mapreduce.framework.name)

All: I am trying to get the Giraph 0.2 snapshot (pulled via GIT on Friday) to build and run with CDH4. I modified the pom.xml to provide a profile for my specific version (4.1.1). The build works (mvn -Phadoop_cdh4.1.1 clean package test) and passes all the tests. If I try to do the next step

Re: Waiting for times required to be 19 (currently 18)

Nate: I am fighting to get Giraph built against CDH 4.1.1. Can you provide the process (mvn flags,) and changes you used to get your build to work. I can get core to build but examples tests fail out. But I can get the page rank example in the core jar to work in some cases. I get a lo

Re: Issue running Giraph with CDH4.1.1.

Eugene: Not a obvious question and I should have mentioned in my post that I have been running a number of other tests and engines on this cluster (Hive, Pig, Mahout, JCuda, etc.) successfully. On 2/14/2013 4:09 PM, Eugene Koontz wrote: Hi David, Please forgive this obvious quest