[
https://issues.apache.org/jira/browse/HDFS-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Pak updated HDFS-6325:
----------------------------
Attachment: HDFS-6325.patch
Here is a patch with a proposed fix to introduce
BlockManager#isSufficientlyReplicated and a unit test.
-This will check that the block is replicated to at least the min of number of
live DNs and the replication factor of the file.
The unit test will
1. start 6 DNs
2. create a file with a rep factor of 3
3. Kill the DNs with the block locations of that file.
4. run append and ensure failure
5. ensure the file is not opened afterwards.
> Append should fail if the last block has insufficient number of replicas
> ------------------------------------------------------------------------
>
> Key: HDFS-6325
> URL: https://issues.apache.org/jira/browse/HDFS-6325
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.2.0
> Reporter: Konstantin Shvachko
> Assignee: Keith Pak
> Attachments: HDFS-6325.patch, HDFS-6325_test.patch, appendTest.patch
>
>
> Currently append() succeeds on a file with the last block that has no
> replicas. But the subsequent updatePipeline() fails as there are no replicas
> with the exception "Unable to retrieve blocks locations for last block". This
> leaves the file unclosed, and others can not do anything with it until its
> lease expires.
> The solution is to check replicas of the last block on the NameNode and fail
> during append() rather than during updatePipeline().
> How many replicas should be present before NN allows to append? I see two
> options:
> # min-replication: allow append if the last block is minimally replicated (1
> by default)
> # full-replication: allow append if the last block is fully replicated (3 by
> default)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
