[
https://issues.apache.org/jira/browse/HDFS-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957019#comment-16957019
]
Ayush Saxena commented on HDFS-13736:
-------------------------------------
Thanx [~xiaodong.hu], Just adding a param and having a overloaded method works
with minimal changes, I think. I tried doing so, the ut passes for me, give a
check :
{code:java}
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
index b50d479c786..8f7e893e1b6 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
@@ -240,9 +240,10 @@ protected void chooseFavouredNodes(String src, int
numOfReplicas,
DatanodeDescriptor favoredNode = favoredNodes.get(i);
// Choose a single node which is local to favoredNode.
// 'results' is updated within chooseLocalNode
- final DatanodeStorageInfo target =
- chooseLocalStorage(favoredNode, favoriteAndExcludedNodes, blocksize,
- maxNodesPerRack, results, avoidStaleNodes, storageTypes, false);
+ final DatanodeStorageInfo target = chooseLocalStorage(favoredNode, true,
+ favoriteAndExcludedNodes, blocksize, maxNodesPerRack, results,
+ avoidStaleNodes, storageTypes);
+
if (target == null) {
LOG.warn("Could not find a target for file " + src
+ " with favored node " + favoredNode);
@@ -546,12 +547,22 @@ protected DatanodeStorageInfo chooseLocalStorage(Node
localMachine,
List<DatanodeStorageInfo> results, boolean avoidStaleNodes,
EnumMap<StorageType, Integer> storageTypes)
throws NotEnoughReplicasException {
+ return chooseLocalStorage(localMachine, false, excludedNodes, blocksize,
+ maxNodesPerRack, results, avoidStaleNodes, storageTypes);
+ }
+
+ protected DatanodeStorageInfo chooseLocalStorage(Node localMachine,
+ boolean isFavoredNode, Set<Node> excludedNodes, long blocksize,
+ int maxNodesPerRack, List<DatanodeStorageInfo> results,
+ boolean avoidStaleNodes, EnumMap<StorageType, Integer> storageTypes)
+ throws NotEnoughReplicasException {
// if no local machine, randomly choose one node
if (localMachine == null) {
return chooseRandom(NodeBase.ROOT, excludedNodes, blocksize,
maxNodesPerRack, results, avoidStaleNodes, storageTypes);
}
- if (preferLocalNode && localMachine instanceof DatanodeDescriptor
+ if ((preferLocalNode || isFavoredNode)
+ && localMachine instanceof DatanodeDescriptor
&& clusterMap.contains(localMachine)) {
DatanodeDescriptor localDatanode = (DatanodeDescriptor) localMachine;
// otherwise try local machine first
@@ -575,7 +586,7 @@ protected DatanodeStorageInfo chooseLocalStorage(Node
localMachine,
return localStorage;
}
}
- }
+ }
}
return null;
}
{code}
If this works, you can update accordingly, adding the annotations as
[~hexiaoqiao] mentioned.
> BlockPlacementPolicyDefault can not choose favored nodes when
> 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-13736
> URL: https://issues.apache.org/jira/browse/HDFS-13736
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.2.0
> Reporter: hu xiaodong
> Assignee: hu xiaodong
> Priority: Major
> Attachments: HDFS-13736.001.patch, HDFS-13736.002.patch,
> HDFS-13736.003.patch, HDFS-13736.004.patch, HDFS-13736.005.patch
>
>
> BlockPlacementPolicyDefault can not choose favored nodes when
> 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
