[ https://issues.apache.org/jira/browse/HDFS-11104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831086#comment-15831086 ]
Doris Gu commented on HDFS-11104: --------------------------------- I've already been working on this, is it possible to assign to me? > BlockPlacementPolicyDefault choose favoredNodes in turn which may cause > imbalance > --------------------------------------------------------------------------------- > > Key: HDFS-11104 > URL: https://issues.apache.org/jira/browse/HDFS-11104 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 3.0.0-alpha1 > Reporter: Doris Gu > Assignee: Ajith S > Attachments: HDFS-11104.patch > > > if client transfer favoredNodes when it writes files into hdfs,chooseTarget > in BlockPlacementPolicyDefault prior chooseTarget in turn: > {quote} > DatanodeStorageInfo[] chooseTarget(String src, > int numOfReplicas, > Node writer, > Set<Node> excludedNodes, > long blocksize, > List<DatanodeDescriptor> favoredNodes, > BlockStoragePolicy storagePolicy) { > try { > ... > *for (int i = 0; i < favoredNodes.size() && results.size() < > numOfReplicas; i++)* { > DatanodeDescriptor favoredNode = favoredNodes.get(i); > // Choose a single node which is local to favoredNode. > // 'results' is updated within chooseLocalNode > final DatanodeStorageInfo target = chooseLocalStorage(favoredNode, > favoriteAndExcludedNodes, blocksize, maxNodesPerRack, > results, avoidStaleNodes, storageTypes, false); > ... > {quote} > why not shuffle it here? Make block more balanced, save the cost balancer > will pay and make cluster more stable. > {quote} > for (DatanodeDescriptor favoredNode : > DFSUtil.shuffle(favoredNodes.toArray(new > DatanodeDescriptor[favoredNodes.size()]))) > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org