On 28.08.2007, at 20:06, Raghu Angadi wrote:
This means that a datanode is asked to receive a new block but the
block already exists.
That's what I thought when I looked at the code ..but why would it
receive such a request? (Especially that often!) Shouldn't the name
node know what blocks are where?
One case where I have seen this happen is when Namenode is trying
to replicate blocks. Say replication increased from 3 to 20,
Namenode asks might ask multiple datanodes to transfer a block to
the same datanode. Smaller the cluster the cluster more probability
that this happens.
What context are you seeing this?
Good question ...we have seen that come and go. Somehow related to
the execution of some of our jobs. We haven't found the exact cause
yet. At some stage we have seen hundreds of these exceptions within a
few minutes. So far we never increased the replication. But even on
our 0.14 test cluster we have seen them now. And it's not like we
have been doing any stress tests on there (yet).
cheers
--
Torsten
