[
https://issues.apache.org/jira/browse/HDFS-15355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108080#comment-17108080
]
Ayush Saxena edited comment on HDFS-15355 at 5/15/20, 8:35 AM:
---------------------------------------------------------------
Thanx [~hadoop_yangyun] for the patch.
{code:java}
+ public static final String DFS_STORAGE_POLICY_DEFAULT ={code}
Change the name, Default in the end appears to be default value for the config,
as done in other configs. Can have similar naming semantics as erasure coding
policy.
In {{hdfs_defaults}} you can specify the value HOT here. and remove the line
below.
{code:java}
+ <value></value>{code}
For the resolution from configuration, you can do like this.
{code:java}
String defaultPolicy =
conf.get(DFSConfigKeys.DFS_STORAGE_POLICY_DEFAULT, "HOT");
for (BlockStoragePolicy policy : policies) {
if (policy!=null && policy.getName().equalsIgnoreCase(defaultPolicy))
{ defaultId = policy.getId();
break; }
}
}
{code}
Instead of explicit {{"HOT"}}, you add a variable in \{{DFSConfigKeys}} ending
with \{{DEFAULT}} and use it.
{code:java}
@VisibleForTesting
public static BlockStoragePolicySuite createDefaultSuite() {
return createDefaultSuite(null);
}
{code}
This method tends to be just for tests? If so, remove it and use the new one
only, passing {{new Configuration()}} in the methods where conf object isn't
already present. Can then remove the null check for conf too in the above
method.
{code:java}
newConf.setBoolean(DFSConfigKeys.DFS_STORAGE_POLICY_ENABLED_KEY, true);
{code}
This by default true, no need to explicitly add.
Regarding the name of variable {{newCluster}} seems not too cool, any problems
keeping it as just cluster?
{code:java}
DFSTestUtil.createFile(newfs, fooFile, 0, REPLICATION, 0L);
{code}
File length you can make 0 and verify your test as above, and make the
datanodes as 0, to speed up the test.
Test failure may be related, Give a check. especially {{TestHdfsConfigFields}}
was (Author: ayushtkn):
Thanx [~hadoop_yangyun] for the patch.
{code:java}+ public static final String DFS_STORAGE_POLICY_DEFAULT =
Change the name, Default in the end appears to be default value for the config,
as done in other configs. Can have similar naming semantics as erasure coding
policy.
In {{hdfs_defaults}} you can specify the value HOT here. and remove the line
below.
{code:java}
+ <value></value>
{code}
For the resolution from configuration, you can do like this.
{code:java}
String defaultPolicy =
conf.get(DFSConfigKeys.DFS_STORAGE_POLICY_DEFAULT, "HOT");
for (BlockStoragePolicy policy : policies) {
if (policy!=null && policy.getName().equalsIgnoreCase(defaultPolicy))
{
defaultId = policy.getId();
break;
}
}
}
{code}
Instead of explicit {{"HOT"}}, you add a variable in {{DFSConfigKeys}} ending
with {{DEFAULT}} and use it.
{code:java}
@VisibleForTesting
public static BlockStoragePolicySuite createDefaultSuite() {
return createDefaultSuite(null);
}
{code}
This method tends to be just for tests? If so, remove it and use the new one
only, passing {{new Configuration()}} in the methods where conf object isn't
already present. Can then remove the null check for conf too in the above
method.
{code:java}
newConf.setBoolean(DFSConfigKeys.DFS_STORAGE_POLICY_ENABLED_KEY, true);
{code}
This by default true, no need to explicitly add.
Regarding the name of variable {{newCluster}} seems not too cool, any problems
keeping it as just cluster?
{code:java}
DFSTestUtil.createFile(newfs, fooFile, 0, REPLICATION, 0L);
{code}
File length you can make 0 and verify your test as above, and make the
datanodes as 0, to speed up the test.
Test failure may be related, Give a check. especially {{TestHdfsConfigFields}}
> Make the default block storage policy ID configurable
> -----------------------------------------------------
>
> Key: HDFS-15355
> URL: https://issues.apache.org/jira/browse/HDFS-15355
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: block placement, namenode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15355.001.patch, HDFS-15355.002.patch,
> HDFS-15355.003.patch
>
>
> Make the default block storage policy ID configurable. Sometime we want to
> use different storage policy ID from the startup of cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
