[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967462#comment-15967462 ]
Yiqun Lin commented on HDFS-11530: ---------------------------------- Thanks [~arpitagarwal] for the suggestion. The comment makes sense to me. The followings are what I did in my latest patch and will help you review. * Add the new setting {{dfs.net.topology.impl}} that only used to choose nodes to place replicas but not used to balance/move block replicas. * The default topology of the setting is {{NetworkTopology}} since we still need more testings to test {{DFSNetworkTopology}} before making it as the default(as [~arpitagarwal] suggested). * Update place that used the old config name from {{CommonConfigurationKeysPublic.NET_TOPOLOGY_IMPL_KEY}} to {{DFSConfigKeys.DFS_NET_TOPOLOGY_IMPL_KEY}} and add an additional test to test the function of the new DFSNetworkTopology which parsed from the new setting. Any comments are welcomed. Thanks. > Use HDFS specific network topology to choose datanode in > BlockPlacementPolicyDefault > ------------------------------------------------------------------------------------ > > Key: HDFS-11530 > URL: https://issues.apache.org/jira/browse/HDFS-11530 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode > Affects Versions: 3.0.0-alpha2 > Reporter: Yiqun Lin > Assignee: Yiqun Lin > Attachments: HDFS-11530.001.patch, HDFS-11530.002.patch, > HDFS-11530.003.patch, HDFS-11530.004.patch, HDFS-11530.005.patch, > HDFS-11530.006.patch, HDFS-11530.007.patch, HDFS-11530.008.patch, > HDFS-11530.009.patch, HDFS-11530.010.patch, HDFS-11530.011.patch > > > The work for {{chooseRandomWithStorageType}} has been merged in HDFS-11482. > But this method is contained in new topology {{DFSNetworkTopology}} which is > specified for HDFS. We should update this and let > {{BlockPlacementPolicyDefault}} use the new way since the original way is > inefficient. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org