[
https://issues.apache.org/jira/browse/HDFS-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995073#comment-14995073
]
Hudson commented on HDFS-9379:
------------------------------
SUCCESS: Integrated in Hadoop-Yarn-trunk #1372 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1372/])
HDFS-9379. Make NNThroughputBenchmark support more than 10 datanodes. (arp: rev
2801b42a7e178ad6a0e6b0f29f22f3571969c530)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Make NNThroughputBenchmark$BlockReportStats support more than 10 datanodes
> --------------------------------------------------------------------------
>
> Key: HDFS-9379
> URL: https://issues.apache.org/jira/browse/HDFS-9379
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9379.000.patch
>
>
> Currently, the {{NNThroughputBenchmark}} test {{BlockReportStats}} relies on
> sorted {{datanodes}} array in the lexicographical order of datanode's
> {{xferAddr}}.
> * There is an assertion of datanode's {{xferAddr}} lexicographical order when
> filling the {{datanodes}}, see [the
> code|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java#L1152].
> * When searching the datanode by {{DatanodeInfo}}, it uses binary search
> against the {{datanodes}} array, see [the
> code|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java#L1187]
> In {{DatanodeID}}, the {{xferAddr}} is defined as {{host:port}}. In
> {{NNThroughputBenchmark}}, the port is simply _the index of the tiny
> datanode_ plus one.
> The problem here is that, when there are more than 9 tiny datanodes
> ({{numThreads}}), the lexicographical order of datanode's {{xferAddr}} will
> be invalid as the string value of datanode index is not in lexicographical
> order any more. For example,
> {code}
> ...
> 192.168.54.40:8
> 192.168.54.40:9
> 192.168.54.40:10
> 192.168.54.40:11
> ...
> {code}
> {{192.168.54.40:9}} is greater than {{192.168.54.40:10}}. The assertion will
> fail and the binary search won't work.
> The simple fix is to calculate the datanode index by port directly, instead
> of using binary search.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
