I checked the input just creating the graph and comparing it. While I cant
say the graph is correct (its to big) its at least consistent.
So the only things where the different output can come from is the
connected component part (see code further down). I'm completely stomped,
the code is basically the example code Giraph ships with all I changed was
replacing the IntWriteable Id's with Text.
Anyone has any Idea what the problem could be, I'm running out of Idea's.
cheers Martin
@Override
public void compute(Vertex<Text, Text, NullWritable> vertex,
Iterable<Text> inmessage) throws IOException {
boolean changed = false;
// first superstep && setup
if (getSuperstep() == 0) {
//initialize value
vertex.setValue(vertex.getId()
);
//cheating by checking the neighbors ID's (cuts down 1
iteration)
for (Edge<Text, NullWritable> e : vertex.getEdges()) {
Text candidate = e.getTargetVertexId();
if (candidate.compareTo(vertex.getValue()) < 0) {
changed = true;
vertex.setValue(candidate);
}
}
}
// other superstep
else {
// read all messages and compare with own state
for (Text message : inmessage) {
if (message.compareTo(vertex.getValue()) < 0) {
changed = true;
vertex.setValue(message);
}
}
}
// if state has changed send a message to all neighbors
if (changed && getSuperstep() < limiter) {
sendMessageToAllEdges(vertex, vertex.getValue());
}
vertex.voteToHalt();
}
On Mon, Mar 3, 2014 at 9:56 AM, Martin Neumann <mneum...@spotify.com> wrote:
> Hey,
>
> sorry, yes I intended to send it to the mailing-list, thanks for the hint
> (I repost it to the list). Guess I should stop working in the middle of the
> night. :-)
>
> I tested the input format and its output is consistent, I always get the
> same graph when running it. So the bug has to be in the connected component
> part since that one gives me different outputs for the same input graph.
>
> cheers Martin
>
>
> On Mon, Mar 3, 2014 at 8:06 AM, Sebastian Schelter <s...@apache.org> wrote:
>
>> Martin,
>>
>> can you write a MapReduce job that creates your graph and run it with a
>> simpler inputformat?
>>
>> I really suspect that the bug lies somewhere in your input format.
>>
>> --sebastian
>>
>>
>> On 03/02/2014 09:48 PM, Martin Neumann wrote:
>>
>>> I checked the input just creating the graph and comparing it. While I
>>> cant
>>> say the graph is correct (for its to big) its at least consistent.
>>>
>>> So the only things where the different output can come from is the
>>> connected component part (see code in the first mail). I'm completely
>>> stomped, the code is basically the example code Giraph ships with all I
>>> changed was replacing the IntWriteable Id's with Text.
>>>
>>> Anyone has any Idea what the problem could be, I'm running out of Idea's.
>>>
>>> cheers Martin
>>>
>>>
>>> On Thu, Feb 27, 2014 at 4:59 PM, Sebastian Schelter <s...@apache.org>
>>> wrote:
>>>
>>> Hi Martin
>>>>
>>>> I don't think that there are problems with comparing and sorting Text
>>>> writables as Hadoop is basically a big external sorting system.
>>>>
>>>> I'm not sure I understand your edge input reader, it looks very complex,
>>>> maybe there's a bug somewhere. You could try to preprocess your data
>>>> using
>>>> Hadoop so that you can use a simple VertexInputFormat and see if your
>>>> problems still occur.
>>>>
>>>> --sebastian
>>>>
>>>>
>>>> On 02/27/2014 04:41 PM, Martin Neumann wrote:
>>>>
>>>> Hm
>>>>>
>>>>> I ran the job 5 times and made a diff between the outputs and they are
>>>>> not
>>>>> the same. I cant find anything in the code that could lead to this
>>>>> behaviour.
>>>>>
>>>>> The only idea where to look a the moment would be the identifier. Has
>>>>> anyone experience with String identifier?
>>>>> Is a possible that there are problems with comparing and sorting
>>>>> TextWritables?
>>>>>
>>>>> cheers Martin
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
