Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Alessandro Presta
Most likely because there was no network I/O, hence no serialization involved. Alessandro From: Zachary Hanif mailto:zh4...@gmail.com>> Reply-To: "user@giraph.apache.org" mailto:user@giraph.apache.org>> Date: Wednesday, February 13, 2013 1:25 PM To: "user@giraph.ap

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Zachary Hanif
Haha! Always good to see my problems resulting from my own foolishness :) Per your advice, I replaced edgeList.add( > new DefaultEdge( > new Text(tokens[i]), > null > ) > );

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Alessandro Presta
edgeList.add( new DefaultEdge( new Text(tokens[i]), null ) ); This is the issue: NullWritable and null are not the same. You should replace null with NullWritable.get(), or e

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Zachary Hanif
Sure thing! Credit where it's due, this is heavily cribbed from https://github.com/castagna/jena-grande/tree/master/src/main/java/org/apache/jena/grande/giraph import java.io.IOException; > import java.util.ArrayList; > import java.util.List; > import java.util.Map; > > import org.apache.giraph.b

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Alessandro Presta
You probably have a pretty small input that is all loaded from the same worker, so this other worker gets 0 input splits. This shouldn't be a problem. From: Zachary Hanif mailto:zh4...@gmail.com>> Reply-To: "user@giraph.apache.org" mailto:user@giraph.apache.org>> D

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Zachary Hanif
Well, this is a bit odd: > 2013-02-13 20:58:45,740 INFO org.apache.giraph.worker.BspServiceWorker: > loadInputSplits: Using 1 thread(s), originally 1 threads(s) for 14 total > splits. > 2013-02-13 20:58:45,742 INFO org.apache.giraph.comm.netty.NettyServer: start: > Using Netty without authentic

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Alessandro Presta
Can you post your VertexInputFormat code? From: Zachary Hanif mailto:zh4...@gmail.com>> Reply-To: "user@giraph.apache.org" mailto:user@giraph.apache.org>> Date: Wednesday, February 13, 2013 12:31 PM To: "user@giraph.apache.org" mailto

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Zachary Hanif
It is my own code. I'm staring at my VertexInputFormat class right now. It extends TextVertexInputFormat. I cannot imagine why a value would not be set for these vertexes, but I'll drop in some code to more stringently ensure value creation. Why would this begin to fail on a distributed deployment

Re: Giraph/Netty issues on a cluster

2013-02-13 Thread Alessandro Presta
Hi Zachary, Are you running one of the examples or your own code? It seems to me that a call to edge.getValue() is returning null, which should never happen. Alessandro From: Zachary Hanif mailto:zh4...@gmail.com>> Reply-To: "user@giraph.apache.org" mailto:user@g

Re: InputFormat for the example SimpleMasterComputeVertex

2013-02-13 Thread Maria Stylianou
Sorry, I didn't set the parameters -mc for the masterCompute and the -wc for the WorkerContext in the execution command. Now, I'm running hadoop jar target/giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleMasterCom

Re: InputFormat for the example SimpleMasterComputeVertex

2013-02-13 Thread Maja Kabiljo
Hi Maria, You also need to set the MasterCompute class, because that's where aggregators get registered (just like it's done in TstBspBasic.testBspMasterCompute where this vertex is used). Maja From: Claudio Martella mailto:claudio.marte...@gmail.com>> Reply-To: "user@giraph.apache.org

Re: InputFormat for the example SimpleMasterComputeVertex

2013-02-13 Thread Claudio Martella
in the latest trunk, this is line 42: double newValue = this.getAggregatedValue(SMC_AGG).get(); What kind of input graph are you using? On Wed, Feb 13, 2013 at 1:25 PM, Maria Stylianou wrote: > I didn't change anything in the code. I took the example as it is. > Actually, the whole error mess

Re: InputFormat for the example SimpleMasterComputeVertex

2013-02-13 Thread Maria Stylianou
I didn't change anything in the code. I took the example as it is. Actually, the whole error message is this one: java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@

Re: InputFormat for the example SimpleMasterComputeVertex

2013-02-13 Thread Claudio Martella
Are you sure that is the right line? I do not see how that would cause an exception, most probably a for loop on messages right after? On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou wrote: > Hey, > > I am trying to run the example SimpleMasterComputeVertex, but no matter > which Input Format