Re: Adding MasterCompute object causes "failed to report status" errors

2012-08-20 Thread KAUSHIK SARKAR
Hi Nick, Please refer to the SimpleMasterComputeWorkerContext class in the attached SimpleMasterComputeVertex.java file (This is from the snapshot of 0.2 that I am using. It is approx. 1 month old. It seems that the WorkerContext class is different from the current svn version. I am not aware if t

RE: cannot run Giraph trunk with Hadoop 2.0.0-alpha

2012-08-20 Thread David Garcia
You can remove this error by recursively removing _bsp folder from the zookeeper file system...and then running the job again. Probably should remove folder from hdfs too. From: Johnny Zhang [xiao...@cloudera.com] Sent: Monday, August 20, 2012 6:59 PM To

Re: cannot run Giraph trunk with Hadoop 2.0.0-alpha

2012-08-20 Thread Johnny Zhang
sorry for wide distribution, I further check the folder '_bsp/_defaultZkManagerDir/job_1344903945125_0032' exists, and it has one sub folder '_bsp/_defaultZkManagerDir/job_1344903945125_0032/_task' and another file inside, so the hdfs file permission should not be a issue. but not sure why Giraph

Re: Adding MasterCompute object causes "failed to report status" errors

2012-08-20 Thread Nick West
I'm a little confused by the examples in SimpleMasterComputeVertex.java. To me it looks like this is a simple example with one vertex and one aggregator with the following behavior: - The vertex gets the value stored in the aggregator and then adds its previous value to it and stores the result

Re: Adding MasterCompute object causes "failed to report status" errors

2012-08-20 Thread KAUSHIK SARKAR
Hi Nick, Are you using WorkerContext to register the aggregator? You need to override the preApplication() method in WorkerContext to register the aggregator and then override the preSuperstep() method to to tell the workers to use the aggregator (the useAggregator() method). Check the MasterCompu

Adding MasterCompute object causes "failed to report status" errors

2012-08-20 Thread Nick West
Hi, I have a giraph application that runs fine; however, when I add a MasterCompute object (definition following) all of the map tasks time out. I have hadoop configured to run with 8 map processes and giraph to use one worker. Here's the definition of the MasterCompute object: class BPMasterC

Re: cannot run Giraph trunk with Hadoop 2.0.0-alpha

2012-08-20 Thread Johnny Zhang
Alessandro: Thanks for reminding me on that. Now I can run the pagerank example successfully, though I still get one zookeeper server related exception. Here is part of the log: 12/08/20 11:56:44 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://

Re: cannot run Giraph trunk with Hadoop 2.0.0-alpha

2012-08-20 Thread Alessandro Presta
Looks like you compiled for hadoop 0.20.203, which had a different API (that's why we have to use Munge). Can you try recompiling with the hadoop_2.0.0 profile? From: Johnny Zhang mailto:xiao...@cloudera.com>> Reply-To: "user@giraph.apache.org" mailto:user@giraph.

cannot run Giraph trunk with Hadoop 2.0.0-alpha

2012-08-20 Thread Johnny Zhang
Hi, all: I am trying to run Giraph trunk with Hadoop 2.0.0-alpha. I am getting below error when I run a page rank example job with 3 workers. # hadoop jar target/giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50