Martin,
can you write a MapReduce job that creates your graph and run it with a
simpler inputformat?
I really suspect that the bug lies somewhere in your input format.
--sebastian
On 03/02/2014 09:48 PM, Martin Neumann wrote:
I checked the input just creating the graph and comparing it. While I cant
say the graph is correct (for its to big) its at least consistent.
So the only things where the different output can come from is the
connected component part (see code in the first mail). I'm completely
stomped, the code is basically the example code Giraph ships with all I
changed was replacing the IntWriteable Id's with Text.
Anyone has any Idea what the problem could be, I'm running out of Idea's.
cheers Martin
On Thu, Feb 27, 2014 at 4:59 PM, Sebastian Schelter <s...@apache.org> wrote:
Hi Martin
I don't think that there are problems with comparing and sorting Text
writables as Hadoop is basically a big external sorting system.
I'm not sure I understand your edge input reader, it looks very complex,
maybe there's a bug somewhere. You could try to preprocess your data using
Hadoop so that you can use a simple VertexInputFormat and see if your
problems still occur.
--sebastian
On 02/27/2014 04:41 PM, Martin Neumann wrote:
Hm
I ran the job 5 times and made a diff between the outputs and they are not
the same. I cant find anything in the code that could lead to this
behaviour.
The only idea where to look a the moment would be the identifier. Has
anyone experience with String identifier?
Is a possible that there are problems with comparing and sorting
TextWritables?
cheers Martin
