[
https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966610#comment-15966610
]
Arpit Agarwal commented on HDFS-11530:
--------------------------------------
Thanks for the great work [~linyiqun] and [~vagarychen].
I think we need some more stress testing/validation of the new network topology
implementation before we make it the default. Here's my suggestion:
{code}
if (DFSConfigKeys.DFS_BLOCK_REPLICATOR_CLASSNAME_DEFAULT.getName()
.equals(conf.get(DFSConfigKeys.DFS_BLOCK_REPLICATOR_CLASSNAME_KEY))) {
networktopology = DFSNetworkTopology.getInstance(conf);
} else {
networktopology = NetworkTopology.getInstance(conf);
}
{code}
Instead of using the new network topology whenever the BlockPlacementPolicy is
{{BlockPlacementPolicyDefault}}, let's add a new configuration setting that
allows choosing the NetworkTopology class. The rest of the changes in this
patch can go in while we continue testing the new topology implementation. At
some point in the future we can change the default via configuration.
> Use HDFS specific network topology to choose datanode in
> BlockPlacementPolicyDefault
> ------------------------------------------------------------------------------------
>
> Key: HDFS-11530
> URL: https://issues.apache.org/jira/browse/HDFS-11530
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0-alpha2
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Attachments: HDFS-11530.001.patch, HDFS-11530.002.patch,
> HDFS-11530.003.patch, HDFS-11530.004.patch, HDFS-11530.005.patch,
> HDFS-11530.006.patch, HDFS-11530.007.patch, HDFS-11530.008.patch,
> HDFS-11530.009.patch
>
>
> The work for {{chooseRandomWithStorageType}} has been merged in HDFS-11482.
> But this method is contained in new topology {{DFSNetworkTopology}} which is
> specified for HDFS. We should update this and let
> {{BlockPlacementPolicyDefault}} use the new way since the original way is
> inefficient.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
