[
https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156877#comment-16156877
]
Kai Zheng commented on HDFS-7859:
---------------------------------
Thanks Sammi for the update. The patch seems in much better form.
1. Do you have any issue to track the TODO work? Will you plan to do it shortly
for BETA1?
{code}
+ /*
+ * TODO check if there is any file or directory using this policy.
+ * If no, remove this erasure coding policy from system permanently.
+ * */
{code}
2. Hmm, I thought "dfs.namenode.ec.policies.enabled" should have been removed
when adding the API to enable/disable policy. Please checkout the relevant
discussions and issues to ensure this if it hasn't happened already.
bq. HDFS configuration file means "hdfs-site.xml". For example, user can define
enabled policies through property "dfs.namenode.ec.policies.enabled"
Copy the relevant codes again:
{code}
+ for (ErasureCodingPolicy p : getPolicies()) {
+ if (p.getName().equals(policyName) ||
+ (p.getSchema().equals(policy.getSchema()) &&
+ p.getCellSize() == policy.getCellSize())) {
+ // If the same policy loaded from fsImage override policy loaded based
+ // on HDFS configuration file
+ LOG.info("The erasure coding policy name " + policy + " loaded from " +
+ "fsImage overrides the one loaded according to HDFS
hdfs-site.xml");
+ }
+ }
{code}
3. Again, could we make the change: ErasureCodingPolicyManagerSection =>
ErasureCodingSection. Also check related names like
loadErasureCodingPolicyManagerSection, saveErasureCodingPolicyManagerSection.
4. The assert message would be "The erasure coding policy saved into and loaded
from fsimage was bad" to make more sense.
{code}
+ assertEquals("The erasure coding policy is not found",
+ targetPolicy, ecPolicy);
{code}
5. Here or elsewhere, please ensure no policy can be DISABLED/REMOVED if it's
used by files, with necessary tests.
Thanks!
> Erasure Coding: Persist erasure coding policies in NameNode
> -----------------------------------------------------------
>
> Key: HDFS-7859
> URL: https://issues.apache.org/jira/browse/HDFS-7859
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: SammiChen
> Labels: hdfs-ec-3.0-must-do
> Attachments: HDFS-7859.001.patch, HDFS-7859.002.patch,
> HDFS-7859.004.patch, HDFS-7859.005.patch, HDFS-7859.006.patch,
> HDFS-7859.007.patch, HDFS-7859.008.patch, HDFS-7859.009.patch,
> HDFS-7859.010.patch, HDFS-7859.011.patch, HDFS-7859.012.patch,
> HDFS-7859.013.patch, HDFS-7859.014.patch, HDFS-7859.015.patch,
> HDFS-7859-HDFS-7285.002.patch, HDFS-7859-HDFS-7285.002.patch,
> HDFS-7859-HDFS-7285.003.patch
>
>
> In meetup discussion with [~zhz] and [~jingzhao], it's suggested that we
> persist EC schemas in NameNode centrally and reliably, so that EC zones can
> reference them by name efficiently.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
