[
https://issues.apache.org/jira/browse/HDFS-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981291#comment-14981291
]
Jing Zhao commented on HDFS-9289:
---------------------------------
bq. In general, if a client misreports GS, does it indicate a likelihood of
misreported numBytes – and therefore we should deny the commitBlock?
Currently NN only depends on the reported length from the client to determine
the block length (not considering lease recovery scenario). So the only check
we can do about the length is the existing one: {{assert block.getNumBytes() <=
commitBlock.getNumBytes()}}.
bq. But it's still a data loss because the data written by the client after
updatePipeline becomes invisible.
Throwing an exception here may not necessarily mean that the data written after
updatePipeline will get lost. In most cases the data can still get recovered
during the lease recovery, considering the replicas have already get persisted
in DataNodes before client sends out the commit/complete request to NN (since
the client has received the last response from the pipeline at that time).
So throwing exception here should be the correct behavior and may not be that
risky.
> check genStamp when complete file
> ---------------------------------
>
> Key: HDFS-9289
> URL: https://issues.apache.org/jira/browse/HDFS-9289
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Chang Li
> Assignee: Chang Li
> Priority: Critical
> Attachments: HDFS-9289.1.patch, HDFS-9289.2.patch, HDFS-9289.3.patch,
> HDFS-9289.4.patch
>
>
> we have seen a case of corrupt block which is caused by file complete after a
> pipelineUpdate, but the file complete with the old block genStamp. This
> caused the replicas of two datanodes in updated pipeline to be viewed as
> corrupte. Propose to check genstamp when commit block
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
