Thanks for replying, Pavan. I figured out that my message Writable (an
ArrayListWritable) needed to call clear() in readFields() before
calling super():
@Override
public void readFields(DataInput in) throws IOException {
clear();
super.readFields(in);
}
This was an 'of course' moment when I realized it was, like other
Writables, being reused. But what I don't understand is why doesn't
ArrayListWritable#readFields() call clear? Isn't this a nasty bug? ...
Oh wait - sure enough:
ArrayListWritable object is not cleared in readFields()
https://issues.apache.org/jira/browse/GIRAPH-740
Thanks again,
matt
On Tue, Sep 23, 2014 at 11:46 AM, Pavan Kumar A pava...@outlook.com wrote:
Can you give more context?
What are the types of messages, patch of your compute method, etc.
You will not receive messages that are not sent, but one thing that can
happen is
-- message can have multiple parameters.
suppose message objects can have 2 parameters
m - a,b
say in m's write(out) you do not handle the case of b = null
m1 sets b
m2 has b=null
then because of incorrect code for m's write() m2 can show b = m1.b
that is because message objects will be re-used when receiving. This is a
Giraph gotcha, because of
object reuse in most iterators.
Thanks
From: m...@matthewcornell.org
Date: Tue, 23 Sep 2014 10:10:48 -0400
Subject: receiving messages that I didn't send
To: user@giraph.apache.org
Hi Folks. I am refactoring my compute() to use a set of ids as its
message type, and in my tests it is receiving a message that it
absolutely did not send. I've debugged it and am at a loss.
Interestingly, I encountered this once before and solved it by
creating a copy of a Writeable instead of re-using it, but I haven't
been able to solve it this time. In general, does this anomalous
behavior indicate a Giraph/Hadoop gotcha'? It's really confounding!
Thank very much -- matt
--
Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34
Dickinson Street, Amherst MA 01002 | matthewcornell.org
--
Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34
Dickinson Street, Amherst MA 01002 | matthewcornell.org
