[
https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847858#comment-15847858
]
Andrew Wang commented on HDFS-7859:
-----------------------------------
I re-read through the history of this JIRA, and it seems like we've debated a
couple times whether it's useful to persist this information, with the current
status of the patch being to only persist user-added policies.
I think this is not very useful as is, and potentially dangerous. We let the
user specify any ECPolicy they want, without much field validation. This means
the user could specify an ID/name that we already use, or an ID/name we might
want to hardcode later. Even with validation, this makes upgrade difficult.
Given that we haven't finished the pluggable EC policy work, we also don't know
what fields might be required to fully specify an EC policy. This patch does
let the user configure different parameters for Reed Solomon, but we already
provide what we think are a good set of hardcoded policies to choose from.
IMO where some persistence would be useful is for HDFS-11314. We'd like to
restrict the set of EC policies that can be used on a cluster, since fault
tolerance depends on the # of nodes and racks. This would be limiting from the
set of hardcoded policies though, rather than adding new policies.
[~drankye], [~zhz], thoughts on this?
> Erasure Coding: Persist erasure coding policies in NameNode
> -----------------------------------------------------------
>
> Key: HDFS-7859
> URL: https://issues.apache.org/jira/browse/HDFS-7859
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Xinwei Qin
> Priority: Blocker
> Labels: BB2015-05-TBR, hdfs-ec-3.0-must-do
> Attachments: HDFS-7859.001.patch, HDFS-7859.002.patch,
> HDFS-7859.004.patch, HDFS-7859.005.patch, HDFS-7859.006.patch,
> HDFS-7859.007.patch, HDFS-7859.008.patch, HDFS-7859.009.patch,
> HDFS-7859-HDFS-7285.002.patch, HDFS-7859-HDFS-7285.002.patch,
> HDFS-7859-HDFS-7285.003.patch
>
>
> In meetup discussion with [~zhz] and [~jingzhao], it's suggested that we
> persist EC schemas in NameNode centrally and reliably, so that EC zones can
> reference them by name efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
