[
https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14692549#comment-14692549
]
Jing Zhao commented on HDFS-8833:
---------------------------------
Thanks for the summary, Zhe. The new proposal looks reasonable to me overall.
Some thoughts and questions:
# Will we allow associating the EC policy with a non-empty directory? I guess
we should disallow it, otherwise the semantic of the "create EC Directory"
command can be very confusing.
# Do we want to allow nested EC directories? Currently since we only support
one policy, I do not see any benefits to have nested EC directories. Thus in
the first stage we can disallow it. Also note that it's always easier to remove
a restriction than adding a new restriction.
# If we agree on the above two, the only change we're proposing here is to
support rename across EC zone boundary. Since the EC policy bit is already on
INodeFile, its implementation can be simple.
I also had some offline discussion about this with [~sureshms], [~szetszwo],
and [~wheat9]. Currently our main concern is still to allow rename can make it
hard for end user to understand the exact semantic and also make the management
hard.
> Erasure coding: store EC schema and cell size in INodeFile and eliminate
> notion of EC zones
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-8833
> URL: https://issues.apache.org/jira/browse/HDFS-8833
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: HDFS-7285
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
>
> We have [discussed |
> https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14357754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14357754]
> storing EC schema with files instead of EC zones and recently revisited the
> discussion under HDFS-8059.
> As a recap, the _zone_ concept has severe limitations including renaming and
> nested configuration. Those limitations are valid in encryption for security
> reasons and it doesn't make sense to carry them over in EC.
> This JIRA aims to store EC schema and cell size on {{INodeFile}} level. For
> simplicity, we should first implement it as an xattr and consider memory
> optimizations (such as moving it to file header) as a follow-on. We should
> also disable changing EC policy on a non-empty file / dir in the first phase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
