[
https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886072#comment-13886072
]
Jing Zhao commented on HDFS-5776:
---------------------------------
Thanks for updating the patch, [~xieliang007] and [~stack].
[~stack], so the latest patch changes setThreadsNumForHedgedReads to private
and aims to make users unable to "change the thread number from client side
dynamically". However, users can still create their own configuration object,
change the configuration for thread pool size, create an DFSClient instance,
and change the thread number? So I think we may want to make it more clean
here. Specifically,
# the first DFSClient who tries to enable the hedged read will initialize the
thread pool (in the DFSClient constructor or in the enable method), so that the
enable can be a real enable
# changing of the thread pool size (if it is necessary) should still go through
a setThreadsNumForHedgedReads method (instead of the constructor of DFSClient),
so that a client cannot silently change the size of the thread pool
Besides, the current patch has not addressed the comment for
enoughNodesForHedgedRead/chooseDataNode.
> Support 'hedged' reads in DFSClient
> -----------------------------------
>
> Key: HDFS-5776
> URL: https://issues.apache.org/jira/browse/HDFS-5776
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-5776-v10.txt, HDFS-5776-v11.txt, HDFS-5776-v12.txt,
> HDFS-5776-v12.txt, HDFS-5776-v2.txt, HDFS-5776-v3.txt, HDFS-5776-v4.txt,
> HDFS-5776-v5.txt, HDFS-5776-v6.txt, HDFS-5776-v7.txt, HDFS-5776-v8.txt,
> HDFS-5776-v9.txt, HDFS-5776.txt
>
>
> This is a placeholder of hdfs related stuff backport from
> https://issues.apache.org/jira/browse/HBASE-7509
> The quorum read ability should be helpful especially to optimize read outliers
> we can utilize "dfs.dfsclient.quorum.read.threshold.millis" &
> "dfs.dfsclient.quorum.read.threadpool.size" to enable/disable the hedged read
> ability from client side(e.g. HBase), and by using DFSQuorumReadMetrics, we
> could export the interested metric valus into client system(e.g. HBase's
> regionserver metric).
> The core logic is in pread code path, we decide to goto the original
> fetchBlockByteRange or the new introduced fetchBlockByteRangeSpeculative per
> the above config items.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
