Edge value types don't match

2013-05-17 Thread Ahmet Emre Aladağ
Hi, I'm trying to implement LinkRank in Giraph. For now, I started with a copy of PageRank example and trying to run and improve it. My VertexInputFormat: Text Float My EdgeInputFormat: Text Text where Text are URLs and Float is the LinkRank score. Now I'm getting a type mismatch error

NullPointerException at the last superstep

2013-05-17 Thread Han JU
Hi, I'm developing my first application based on Giraph 1.0.0. I make it run on our cluster. It's a job with fixed number of supersteps. Everything runs smoothly until the last superstep, this Exception got caught in every worker: java.lang.IllegalStateException: run: Caught an unrecoverable

What if the resulting graph is larger than the memory?

2013-05-17 Thread Han JU
Hi, It's me again. After a day's work I've coded a Giraph solution for my problem at hand. I gave it a run on a medium dataset and it's notably faster than other approaches. However the goal is to process larger inputs, for example I've a larger dataset that the result graph is about 400GB when

Re: Edge value types don't match

2013-05-17 Thread Alessandro Presta
This looks like a bug. I assume you're using a VertexValueInputFormat, which is really a VertexInputFormat with E = Writable. The problem is that we're checking that against the E parameter in Vertex, which is NullWritable. VertexValueInputFormat was modified after GiraphTypeValidator was created,

Re: What if the resulting graph is larger than the memory?

2013-05-17 Thread Maja Kabiljo
Hi JU, One thing you can try is to use out-of-core graph (giraph.useOutOfCoreGraph option). I don't know what your exact use case is – do you have the graph which is huge or the data which you calculate in your application is? In the second case, there is 'giraph.doOutputDuringComputation'