[
https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054044#comment-14054044
]
Suresh Srinivas edited comment on HDFS-6482 at 7/7/14 7:10 PM:
---------------------------------------------------------------
bq. The new "current" directory will contain hardlinks to the block and
metadata files in the "previous" directory. So it seems like rollback should
work fine in this case.
Can a brief design doc be posted to this jira to describe what the new
directory structure is, what happens during upgrade to release with this
change? I do not have time to review the patch to figure this out.
My read is, this will not work if during upgrade the previous layout is changed
to the new layout. During rolling upgrades hardlinks to all the blocks are
*not* created, only for the ones deleted post rolling upgrade. This is done to
keep the datanode upgrade time short to support quick restart. If rolling
upgrades cannot be supported, this code can only go into a major release.
was (Author: sureshms):
bq. The new "current" directory will contain hardlinks to the block and
metadata files in the "previous" directory. So it seems like rollback should
work fine in this case.
Can a brief design doc be posted to this jira to describe what the new
directory structure is, what happens during upgrade to release with this
change? I do not have time to review jira to figure this out.
My read is, this will not work if during upgrade the previous layout is changed
to the new layout. During rolling upgrades hardlinks to all the blocks are
*not* created, only for the ones deleted post rolling upgrade. This is done to
keep the datanode upgrade time short to support quick restart. If rolling
upgrades cannot be supported, this code can only go into a major release.
> Use block ID-based block layout on datanodes
> --------------------------------------------
>
> Key: HDFS-6482
> URL: https://issues.apache.org/jira/browse/HDFS-6482
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.5.0
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6482.1.patch, HDFS-6482.2.patch, HDFS-6482.3.patch,
> HDFS-6482.4.patch, HDFS-6482.5.patch, HDFS-6482.6.patch, HDFS-6482.7.patch,
> HDFS-6482.patch
>
>
> Right now blocks are placed into directories that are split into many
> subdirectories when capacity is reached. Instead we can use a block's ID to
> determine the path it should go in. This eliminates the need for the LDir
> data structure that facilitates the splitting of directories when they reach
> capacity as well as fields in ReplicaInfo that keep track of a replica's
> location.
> An extension of the work in HDFS-3290.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
