[
https://issues.apache.org/jira/browse/HDFS-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898246#comment-15898246
]
Andrew Wang commented on HDFS-11314:
------------------------------------
Hi Rakesh, thanks for reviewing!
bq. I'd also prefer keep the list empty and lets users configure the right
values based on their topology.
Great, I filed HDFS-11505. I'll handle this after we get this one in.
bq. Could you please point me the way Namenode handles this during restarts
and read from edit log. It would be good to add test case if not added.
I extended TestErasureCodingPolicies#testBasicSetECPolicy, little snippets here:
{code}
+ // Verify that policies are successfully loaded even when policies
+ // are disabled
+ cluster.getConfiguration(0).set(
+ DFSConfigKeys.DFS_NAMENODE_EC_POLICIES_ENABLED_KEY, "");
+ cluster.restartNameNodes();
+ cluster.waitActive();
{code}
{code}
+ // Also check loading disabled EC policies from fsimage
+ fs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_ENTER);
+ fs.saveNamespace();
+ fs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_LEAVE);
+ cluster.restartNameNodes();
{code}
bq. Whats your opinion to provide a facility to dynamically/reconfigure
enabling the policies with out needing to restart Namenode?
I think changing the set of EC policies is a pretty rare operation, so it's not
that important.
One related nice work item would be doing this for the replication throttles,
since that's an issue even now.
> Enforce set of enabled EC policies on the NameNode
> --------------------------------------------------
>
> Key: HDFS-11314
> URL: https://issues.apache.org/jira/browse/HDFS-11314
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Affects Versions: 3.0.0-alpha1
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Priority: Blocker
> Labels: hdfs-ec-3.0-must-do
> Attachments: HDFS-11314.001.patch, HDFS-11314.002.patch,
> HDFS-11314.003.patch
>
>
> Filing based on discussion in HDFS-8095. A user might specify a policy that
> is not appropriate for the cluster, e.g. a RS (10,4) policy when the cluster
> only has 10 nodes. The NN should only allow the client to choose from a
> pre-approved list determined by the cluster administrator.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
