[
https://issues.apache.org/jira/browse/HDFS-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438387#comment-13438387
]
Colin Patrick McCabe commented on HDFS-3731:
--------------------------------------------
bq. Colin, the approach and your patch looks good to me. What's the latest on
testing? Eg you verified a v1 install with hsync files correctly upgraded (eg
leases recovered) to v2 or trunk build with this patch?
I installed branch-1 and started a one-node cluster. I have a small test
program which creates a file, calls {{hsync}} on it, and then sleeps for a few
minutes. During its sleep, I brought down the cluster. Then I had a block in
blocksBeingWritten corresponding to the last block of that file that was
hsync'ed. (I guess it's called {{sync}} in branch-1).
Then installed a hacked version of trunk running this patch. The hack was to
decrease the lease hard recovery time. It is normally hard-coded to be an hour
and I didn't want to wait that long.
I ran {{./bin/start-dfs.sh -upgrade}}. I confirmed that the last block of the
file was recovered and moved into {{rbw}}, then into {{finalized/}} and then
could be read by me.
Then I stopped the cluster, ran ./bin/namenode -finalize, answered 'Y', and
restarted the cluster. The {{blocksBeingWritten}} directory was gone, as well
as the {{previous}} directory.
The unit test does basically the same thing, in a more systematic way. Hope
this helps.
> 2.0 release upgrade must handle blocks being written from 1.0
> -------------------------------------------------------------
>
> Key: HDFS-3731
> URL: https://issues.apache.org/jira/browse/HDFS-3731
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 2.0.0-alpha
> Reporter: Suresh Srinivas
> Assignee: Colin Patrick McCabe
> Priority: Blocker
> Attachments: HDFS-3731.002.patch, HDFS-3731.003.patch
>
>
> Release 2.0 upgrades must handle blocks being written to (bbw) files from 1.0
> release. Problem reported by Brahma Reddy.
> The {{DataNode}} will only have one block pool after upgrading from a 1.x
> release. (This is because in the 1.x releases, there were no block pools--
> or equivalently, everything was in the same block pool). During the upgrade,
> we should hardlink the block files from the {{blocksBeingWritten}} directory
> into the {{rbw}} directory of this block pool. Similarly, on {{-finalize}},
> we should delete the {{blocksBeingWritten}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
