[
https://issues.apache.org/jira/browse/HADOOP-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654946#action_12654946
]
Hairong Kuang commented on HADOOP-4702:
---------------------------------------
Ant test-core was successful:
BUILD SUCCESSFUL
Total time: 127 minutes 28 seconds
Ant test-patch result:
[exec] +1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] +1 tests included. The patch appears to include 5 new or
modified tests.
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath
integrity.
> Failed block replication leaves an incomplete block in receiver's tmp data
> directory
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-4702
> URL: https://issues.apache.org/jira/browse/HADOOP-4702
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.2
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: tmpBlockRemoval.patch, tmpBlockRemoval1.patch,
> tmpBlockRemoval2.patch
>
>
> When a failure occurs while replicating a block from a source DataNode to a
> target DataNode, the target node keeps an incomplete on-disk copy of the
> block in its temp data directory and an in-memory copy of the block in
> ongoingCreates queue. This causes two problems:
> 1. Since this block is not (should not) be finalized, NameNode is not aware
> of the existence of this incomplete block. It may schedule replicating the
> same block to this node again, which will fail with a message: "Block XX has
> already been started (though not completed), and thus cannot be created."
> 2. Restarting the datanode moves the blocks under the temp data directory to
> be valid blocks, thus introduces corrupted blocks into HDFS. Sometimes those
> corrupted blocks stay in the system undetected if it happens that the partial
> block and its checksums match.
> A failed block replication should clean up both the in-memory & on-disk
> copies of the incomplete block.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
