[
https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088369#comment-15088369
]
Kai Zheng commented on HDFS-8999:
---------------------------------
This isn't a review but just want to understand the work. Some minor comments
or questions by the way. Thanks for clarifying.
Regarding the following codes:
{code}
- if (lastBlock != null && lastBlock.isComplete()
+ if (lastBlock != null) {
+ if (lastBlock.getBlockUCState() == BlockUCState.COMMITTED) {
+ throw new BlockNotYetCompleteException("append: lastBlock="
+ + lastBlock + " of src=" + path
+ + " is COMMITTED but not yet COMPLETE.");
+ } else if (lastBlock.isComplete()
&& !blockManager.isSufficientlyReplicated(lastBlock)) {
- throw new IOException("append: lastBlock=" + lastBlock + " of src="
- + path + " is not sufficiently replicated yet.");
+ throw new IOException("append: lastBlock=" + lastBlock + " of src="
+ + path + " is not sufficiently replicated yet.");
+ }
{code}
* {{lastBlock.isComplete}} means the block has minimal replicas. From the
header comment of {{isSufficientlyReplicated}}, it also means the block has
minimal replicas. Actually it does check if the block has minimal live or
available replicas. So suggest refine the method comment and the IOException
message to avoid confusion here.
* Would we have a method like {{isCommitted}} similar to {{isComplete}}?
* In comments for {{BlockUCState.COMPLETE}}, it says when a block is in this
state then it's not going to be modified. Is this out of sync since a complete
block can append?
> Namenode need not wait for {{blockReceived}} for the last block before
> completing a file.
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-8999
> URL: https://issues.apache.org/jira/browse/HDFS-8999
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Jitendra Nath Pandey
> Assignee: Tsz Wo Nicholas Sze
> Attachments: h8999_20151228.patch, h8999_20160106.patch,
> h8999_20160106b.patch, h8999_20160106c.patch
>
>
> This comes out of a discussion in HDFS-8763. Pasting [~jingzhao]'s comment
> from the jira:
> {quote}
> ...whether we need to let NameNode wait for all the block_received msgs to
> announce the replica is safe. Looking into the code, now we have
> # NameNode knows the DataNodes involved when initially setting up the
> writing pipeline
> # If any DataNode fails during the writing, client bumps the GS and
> finally reports all the DataNodes included in the new pipeline to NameNode
> through the updatePipeline RPC.
> # When the client received the ack for the last packet of the block (and
> before the client tries to close the file on NameNode), the replica has been
> finalized in all the DataNodes.
> Then in this case, when NameNode receives the close request from the client,
> the NameNode already knows the latest replicas for the block. Currently the
> checkReplication call only counts in all the replicas that NN has already
> received the block_received msg, but based on the above #2 and #3, it may be
> safe to also count in all the replicas in the
> BlockUnderConstructionFeature#replicas?
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
