[
https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699965#comment-14699965
]
Andrew Wang commented on HDFS-8833:
-----------------------------------
I mentioned a [possible
design|https://issues.apache.org/jira/browse/HDFS-8833?focusedCommentId=14649618&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14649618]
earlier that unifies SP and EC semantics, it basically comes down to the
default policy on file creation being either "inherit" vs. "set". SP also can
be set on files, so mixed policies within a directory can still happen. I don't
know how confusing mixing is, since replication and SP for instance are already
per-file, and EC feels a lot more like those two than directory-level policies
like snapshots and encryption zones.
We also had a similar question come up during SP design, of if we needed
per-file SP settings at all. At the time Jing convinced me we needed the
flexibility since it's useful for automatic storage tiering. Said system would
automatically adjust things like replication, SP, EC, etc based on usage
information. In this situation it's not okay to require a rename since it's not
transparent to the user.
> Erasure coding: store EC schema and cell size in INodeFile and eliminate
> notion of EC zones
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-8833
> URL: https://issues.apache.org/jira/browse/HDFS-8833
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: HDFS-7285
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-8833-HDFS-7285-merge.00.patch
>
>
> We have [discussed |
> https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14357754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14357754]
> storing EC schema with files instead of EC zones and recently revisited the
> discussion under HDFS-8059.
> As a recap, the _zone_ concept has severe limitations including renaming and
> nested configuration. Those limitations are valid in encryption for security
> reasons and it doesn't make sense to carry them over in EC.
> This JIRA aims to store EC schema and cell size on {{INodeFile}} level. For
> simplicity, we should first implement it as an xattr and consider memory
> optimizations (such as moving it to file header) as a follow-on. We should
> also disable changing EC policy on a non-empty file / dir in the first phase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
