[
https://issues.apache.org/jira/browse/HDFS-11461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886922#comment-15886922
]
Arpit Agarwal edited comment on HDFS-11461 at 2/28/17 12:40 AM:
----------------------------------------------------------------
[~hanishakoneru], a few comments:
# {{DataNode#shutdown}} should use slowDiskDetectionThread.join() instead of
sleep.
# {{diskOutliers}} should maintain the mean read/write/meta latency for each
flagged disk.
# The low threshold should be higher than 1ms (average seek latency of a 7200
RPM disk is 4ms). Let's conservatively set this to 20ms.
# startDiskOutlierDetectionThread should call
Thread.currentThread().interrupt() after catching InterruptedException. See
https://www.ibm.com/developerworks/library/j-jtp05236/
# slowDiskDetectionThread should be a daemon thread.
# DataNodePeerMetrics should also use OutlierDetector constructor that accepts
minNumResources, and pass {{10}}, to keep the behavior consistent with what we
have.
was (Author: arpitagarwal):
[~hanishakoneru], a few comments:
# {{DataNode#shutdown}} should use slowDiskDetectionThread.join() instead of
sleep.
# {{diskOutliers}} should maintain the mean read/write/meta latency for each
flagged disk.
# The low threshold should be higher than 1ms (seek latency of a 7200 RPM disk
is 4ms). Let's conservatively set this to 20ms.
# startDiskOutlierDetectionThread should call
Thread.currentThread().interrupt() after catching InterruptedException. See
https://www.ibm.com/developerworks/library/j-jtp05236/
# slowDiskDetectionThread should be a daemon thread.
# DataNodePeerMetrics should also use OutlierDetector constructor that accepts
minNumResources, and pass {{10}}, to keep the behavior consistent with what we
have.
> DataNode Disk Outlier Detection
> -------------------------------
>
> Key: HDFS-11461
> URL: https://issues.apache.org/jira/browse/HDFS-11461
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
> Reporter: Hanisha Koneru
> Assignee: Hanisha Koneru
> Attachments: HDFS-11461.000.patch
>
>
> Similar to how DataNodes collect peer performance statistics, we can collect
> disk performance statistics per datanode and detect outliers among them, if
> any.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
