[
https://issues.apache.org/jira/browse/HDFS-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141859#comment-14141859
]
Uma Maheswara Rao G commented on HDFS-6995:
-------------------------------------------
Thanks Vinay for working on this.
I am reviewing this patch. Please find a comment on this.
When the client node is not part of any rack configured, then it will be trated
as default rack. What if no node found with default rack we assigned all nodes
with some racks.
In this case, node will try to get from clusterMap with the specified scope of
default rack. But as we don't have any node exist we may get them as null.
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:516)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:481)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseLocalRack(BlockPlacementPolicyDefault.java:411)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseLocalStorage(BlockPlacementPolicyDefault.java:372)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:271)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:212)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:121)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:136)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1504)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3033)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:619)
at
org.apache.hadoop.hdfs.server.namenode.TestBlockPlacementPolicy.testLocalRackPlacement(TestBlockPlacementPolicy.java:72)
{noformat}
Before due to local node null, it was going to chooseRandom by passing scope as
ROOT. So, no issue before this hange in this scenario.
So, if we can't choose any random node with the scope of client nodes rack
scope, then we can consider writer as null itself. So, we can choose random
with ROOT scope.?
I will continue reviewing and post further comments later today if any.
> Block should be placed in the client's 'rack-local' node if 'client-local'
> node is not available
> ------------------------------------------------------------------------------------------------
>
> Key: HDFS-6995
> URL: https://issues.apache.org/jira/browse/HDFS-6995
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.5.0
> Reporter: Vinayakumar B
> Assignee: Vinayakumar B
> Attachments: HDFS-6995-001.patch, HDFS-6995-002.patch,
> HDFS-6995-003.patch, HDFS-6995-004.patch
>
>
> HDFS cluster is rack aware.
> Client is in different node than of datanode,
> but Same rack contains one or more datanodes.
> In this case first preference should be given to select 'rack-local' node.
> Currently, since no Node in clusterMap corresponds to client's location,
> blockplacement policy choosing a *random* node as local node and proceeding
> for further placements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
