[
https://issues.apache.org/jira/browse/HDFS-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994737#comment-13994737
]
Konstantin Shvachko commented on HDFS-6325:
-------------------------------------------
Here are some thoughts after experimenting and discussions.
# We should still allow executing append if the number of live DNs on the
cluster is less than file replication factor. This is to support cases with
small clusters or replication being set intentionally high.
# We should probably go for option (1) with min-replication. This has an
advantage that there will be no change in semantics with current
implementation. The disadvantage is that once file is opened for append the
ongoing replications of the last block will fail, because of the genStamp
change.
# In the end our main goal is to make append fail earlier to prevent the file
from being under construction while nobody can use it, which happens when the
last block has 0 replicas.
> Append should fail if the last block has insufficient number of replicas
> ------------------------------------------------------------------------
>
> Key: HDFS-6325
> URL: https://issues.apache.org/jira/browse/HDFS-6325
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.2.0
> Reporter: Konstantin Shvachko
> Assignee: Keith Pak
> Attachments: HDFS-6325.patch, HDFS-6325_test.patch, appendTest.patch
>
>
> Currently append() succeeds on a file with the last block that has no
> replicas. But the subsequent updatePipeline() fails as there are no replicas
> with the exception "Unable to retrieve blocks locations for last block". This
> leaves the file unclosed, and others can not do anything with it until its
> lease expires.
> The solution is to check replicas of the last block on the NameNode and fail
> during append() rather than during updatePipeline().
> How many replicas should be present before NN allows to append? I see two
> options:
> # min-replication: allow append if the last block is minimally replicated (1
> by default)
> # full-replication: allow append if the last block is fully replicated (3 by
> default)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
