[
https://issues.apache.org/jira/browse/HDFS-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881739#action_12881739
]
sam rash commented on HDFS-1263:
--------------------------------
several thoughts/comments:
my reading of the code is that the temp file was to make the creating of a meta
file that is both truncated and has the new genstamp and atomic operation on
the filesystem. If we rename first and crash and recover, then how do we know
that the truncation didn't finish? (without information from the NN or other
node giving us a new length). If we truncate first, then we have effectively
corrupted a block.
can you also explain the error state that results? (truncated blocks, infinite
loops, bad meta-data, etc)
and do i follow that a client started 2 lease recoveries? or was this a client
and a NN somehow? (ie, how were there concurrent recoveries of the same block)
seems like extra synchronization in parts of updateBlock might help as well.
also, we check the genstamp is moving upwards both at the start of updateBlock
and at the end of tryUpdateBlock. do you know why?
> 0.20: in tryUpdateBlock, the meta file is renamed away before genstamp
> validation is done
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-1263
> URL: https://issues.apache.org/jira/browse/HDFS-1263
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 0.20-append
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.20-append
>
>
> Saw an issue where multiple datanodes are trying to recover at the same time,
> and all of them failed. I think the issue is in FSDataset.tryUpdateBlock, we
> do the rename of blk_B_OldGS to blk_B_OldGS_tmpNewGS and *then* check that
> the generation stamp is moving upwards. Because of this, invalid update block
> calls are blocked, but they then cause future updateBlock calls to fail with
> "Meta file not found" errors.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
