[
https://issues.apache.org/jira/browse/HDFS-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955433#comment-16955433
]
Ayush Saxena commented on HDFS-14882:
-------------------------------------
Thanx [~hexiaoqiao] for the patch. The overall idea looks good. One doubt in
the following logic, if you can help me understand :
{code:java}
for (int start = 0, end = 0; start < activeLen & end < activeLen;) {
if (distances[start] == distances[end]) {
end = end + 1;
if (end < activeLen) continue;
}
Arrays.sort(datanodes, start, end,
Comparator.comparingInt(DatanodeInfo::getXceiverCount));
start = end;
end = end + 1;
}
}{code}
* The first iteration would be start=0 and end=0; the {{ if (distances[start]
== distances[end])}} will always be true, why I don't start with end as 1?
* now at second iteration; start=0 and end=1; distance[0]!= distance[1],
condition tends to be false; then why do I execute {{ Arrays.sort(datanodes,
start, end,
Comparator.comparingInt(DatanodeInfo::getXceiverCount));}}
* Third iter; start = 1; end= 2; if distance isn't equal again we do {{
Arrays.sort(datanodes, start, end,
Comparator.comparingInt(DatanodeInfo::getXceiverCount));}}
Well I need to recheck this logic
You even need to add the new config in {{Hdfs-defaults.xml}}
Apart almost LGTM
> Consider DataNode load when #getBlockLocation
> ---------------------------------------------
>
> Key: HDFS-14882
> URL: https://issues.apache.org/jira/browse/HDFS-14882
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Xiaoqiao He
> Assignee: Xiaoqiao He
> Priority: Major
> Attachments: HDFS-14882.001.patch, HDFS-14882.002.patch
>
>
> Currently, we consider load of datanode when #chooseTarget for writer,
> however not consider it for reader. Thus, the process slot of datanode could
> be occupied by #BlockSender for reader, and disk/network will be busy
> workload, then meet some slow node exception. IIRC same case is reported
> times. Based on the fact, I propose to consider load for reader same as it
> did #chooseTarget for writer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
