[
https://issues.apache.org/jira/browse/HDFS-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356904#comment-14356904
]
Kai Zheng commented on HDFS-7068:
---------------------------------
Thanks all for the good discussion here. We had an offline discussion with
[~walter.k.su].
1. It was thought without introducing EC related features there might not be
multiple file statuses to justify multiple block placement policies, therefore
it would be good to rebase this issue to the EC branch. It's already done,
thanks.
2. We might need to extend existing storage policy concept to allow EC and
stripping cases. If so each file/folder would have an extended storage policy
associated either in inode or xattr, which can be used to get or tell: 1) is
the file in replication mode or stripping ec mode, or pure ec mode; 2) if it's
in ec related mode, then what's the ec schema; 3) if it's in replication mode
by default, then what's the original storage policy in HSM. With such extended
storage policy setting, this work will decide which block placement policy or
policies to use. Existing storage policy is only used in block placement policy
logic, but not used to decide with one to use.
> Support multiple block placement policies
> -----------------------------------------
>
> Key: HDFS-7068
> URL: https://issues.apache.org/jira/browse/HDFS-7068
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 2.5.1
> Reporter: Zesheng Wu
> Assignee: Walter Su
> Attachments: HDFS-7068.patch
>
>
> According to the code, the current implement of HDFS only supports one
> specific type of block placement policy, which is BlockPlacementPolicyDefault
> by default.
> The default policy is enough for most of the circumstances, but under some
> special circumstances, it works not so well.
> For example, on a shared cluster, we want to erasure encode all the files
> under some specified directories. So the files under these directories need
> to use a new placement policy.
> But at the same time, other files still use the default placement policy.
> Here we need to support multiple placement policies for the HDFS.
> One plain thought is that, the default placement policy is still configured
> as the default. On the other hand, HDFS can let user specify customized
> placement policy through the extended attributes(xattr). When the HDFS choose
> the replica targets, it firstly check the customized placement policy, if not
> specified, it fallbacks to the default one.
> Any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
