[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934622#comment-14934622 ]
Ming Ma commented on HDFS-8647: ------------------------------- Thanks [~brahmareddy]! Here are couple comments. * NetworkTopology knows when the # of racks changes from 1 to 2. So instead of having DatanodeManager call {{setHasClusterEverBeenMultiRack}} on NetworkTopology, perhaps we can have a new method {{boolean NetworkTopology.add(Node node)}} where it return true only when the # of racks changes from 1 to 2. Then DatanodeManager can act only if the returned value is true. * {{verifyBlockPlacement}}'s parameter changes from {{LocatedBlock}} to {{DatanodeInfo[]}}. LocatedBlock has other info such as storage type that DatanodeInfo doesn't have. If later we decide to use storage type to verify block placement, we will need to add it back. Any way BlockManager can construct LocatedBlock from the block id? For example, maybe it can get the DatanodeStorageInfo from {{blocksMap.getStorages(block)}} and then construct the block via {{newLocatedBlock}} method. * In {{verifyBlockPlacement}}'s {{if (!clusterMap.hasClusterEverBeenMultiRack() && numRacks <= 1)}}, is {{numRacks <= 1}} still needed? * {{chooseReplicaToDelete}} becomes an internal method. So there is no need to make it public. > Abstract BlockManager's rack policy into BlockPlacementPolicy > ------------------------------------------------------------- > > Key: HDFS-8647 > URL: https://issues.apache.org/jira/browse/HDFS-8647 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Ming Ma > Assignee: Brahma Reddy Battula > Attachments: HDFS-8647-001.patch, HDFS-8647-002.patch, > HDFS-8647-003.patch, HDFS-8647-004.patch, HDFS-8647-004.patch > > > Sometimes we want to have namenode use alternative block placement policy > such as upgrade domains in HDFS-7541. > BlockManager has built-in assumption about rack policy in functions such as > useDelHint, blockHasEnoughRacks. That means when we have new block placement > policy, we need to modify BlockManager to account for the new policy. Ideally > BlockManager should ask BlockPlacementPolicy object instead. That will allow > us to provide new BlockPlacementPolicy without changing BlockManager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)