[
https://issues.apache.org/jira/browse/HDFS-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233895#comment-14233895
]
Ming Ma commented on HDFS-7433:
-------------------------------
This is a good improvement.
Perhaps it is better call {{d.decommissioningStatus.setLastScan(scanNumber);}}
for all nodes, not just decomm-in-progress nodes. Otherwise, the NN could have
nodes with much smaller number than the current scan number after some time. If
you decomm those nodes later, decomm check might keep checking the same set of
nodes until those nodes' scan numbers are the same as the current scan number.
> DatanodeManager#datanodeMap should be a HashMap, not a TreeMap, to optimize
> lookup performance
> ----------------------------------------------------------------------------------------------
>
> Key: HDFS-7433
> URL: https://issues.apache.org/jira/browse/HDFS-7433
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HDFS-7433.patch, HDFS-7433.patch
>
>
> The datanode map is currently a {{TreeMap}}. For many thousands of
> datanodes, tree lookups are ~10X more expensive than a {{HashMap}}.
> Insertions and removals are up to 100X more expensive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
