[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954827#comment-14954827 ]
Vinayakumar B commented on HDFS-9044: ------------------------------------- Hi [~andreina], thanks for the patch, Patch looks almost good. Here are few minor comments, 1. Below lines from {{BlockPlacementPolicyWithNodeGroup#chooseFavouredNodes(..)}} could be just replaced with {{super.chooseFavouredNodes(..)}} {code}+ for (int i = 0; i < favoredNodes.size() && results.size() < numOfReplicas; + i++) { + DatanodeDescriptor favoredNode = favoredNodes.get(i); + // Choose a single node which is local to favoredNode. + // 'results' is updated within chooseLocalNode + DatanodeStorageInfo target = null; + try { + target = + chooseLocalStorage(favoredNode, favoriteAndExcludedNodes, + blocksize, maxNodesPerRack, results, avoidStaleNodes, + storageTypes, false); + } catch (NotEnoughReplicasException e) { + // catch Exception and continue with other favored nodes + continue; + } + if (target == null) { + LOG.warn("Could not find a target for file " + src + + " with favored node " + favoredNode); + continue; + } + favoriteAndExcludedNodes.add(target.getDatanodeDescriptor()); + }{code} 2. {{checkFavoredNodePartOfTarget(..)}} could be renamed to {{isNodeChosen(..)}} 3. {{clusterMapNodeGroup.getNodeGroup(favoredNode.getNetworkLocation())}} can be extracted to a variable {{scope}} for better readability 4. I think its better to update javadoc for {{BlockPlacementPolicyWithNodeGroup#chooseLocalStorage(..)}} by mentioning, fallback to nodegroup/rack will happen if flag {{fallbackToNodeGroupAndLocalRack}} is set. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -------------------------------------------------------------------------------------- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: J.Andreina > Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)