[
https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886107#comment-13886107
]
stack commented on HDFS-5776:
-----------------------------
[~jingzhao] Thanks for the new input. Please help me better understand what
you mean by making more clean so we can adjust the patch accordingly.
Hedged reads are set on or off in the client configuration xml and per
DFSClient instance can be enabled/disabled as you go. Yes, you could read code
and figure that it is possible to do some heavyweight gymnastics creating your
own Configuration -- expensive -- and a new DFSClient -- ditto -- if you wanted
to work around whatever is out in the configuration xml. That seems fine by me
especially as there is no real means of shutting down this access route.
Pardon me but I do not follow what you are asking for in 1. Maybe you are
referring to a 'hole' where if the thread count is <= 0 on construction, the
enable will have no effect -- and you want it to have an 'effect' post
construction?
For 2., you are suggesting that setThreadsNumForHedgedReads not be private but
be available API for the DFSClient to toggle as it sees fit?
I'll let @liang xie address your enoughNodesForHedgedRead comment.
Thanks for checking back.
> Support 'hedged' reads in DFSClient
> -----------------------------------
>
> Key: HDFS-5776
> URL: https://issues.apache.org/jira/browse/HDFS-5776
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-5776-v10.txt, HDFS-5776-v11.txt, HDFS-5776-v12.txt,
> HDFS-5776-v12.txt, HDFS-5776-v2.txt, HDFS-5776-v3.txt, HDFS-5776-v4.txt,
> HDFS-5776-v5.txt, HDFS-5776-v6.txt, HDFS-5776-v7.txt, HDFS-5776-v8.txt,
> HDFS-5776-v9.txt, HDFS-5776.txt
>
>
> This is a placeholder of hdfs related stuff backport from
> https://issues.apache.org/jira/browse/HBASE-7509
> The quorum read ability should be helpful especially to optimize read outliers
> we can utilize "dfs.dfsclient.quorum.read.threshold.millis" &
> "dfs.dfsclient.quorum.read.threadpool.size" to enable/disable the hedged read
> ability from client side(e.g. HBase), and by using DFSQuorumReadMetrics, we
> could export the interested metric valus into client system(e.g. HBase's
> regionserver metric).
> The core logic is in pread code path, we decide to goto the original
> fetchBlockByteRange or the new introduced fetchBlockByteRangeSpeculative per
> the above config items.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
