[
https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877154#comment-13877154
]
Liang Xie commented on HDFS-5776:
---------------------------------
bq. Let's keep TestPread as a test of just pread, and have a separate test to
test hedged reads.
this's what i did, i only changed dfsPreadTest a bit, no creating new conf
object, but repy on the input parameter, that's all, i do have separate cases
for hedged reads, see testHedgedPreadDFSBasic and testMaxOutHedgedReadPool
bq. Could we rename this to getFirstToComplete or something like that? getFirst
just sounds like it's getting the first element in the ArrayList.
OK
bq. I think that when the data being read is local, you will not want hedged
reads. Let's check for this case.
seems you have a different understanding with it, and the latter comments "I
guess we probably don't need to special-case local reads, as long as we
continue to prefer to read from local datanodes when possible" is correct,
there's is no conflict between local read and hedged read. The hedged read
still try to request local read if possible, if no successful until the timeout
reach, then request to the picked secondary dn, then wait the winner.
> Support 'hedged' reads in DFSClient
> -----------------------------------
>
> Key: HDFS-5776
> URL: https://issues.apache.org/jira/browse/HDFS-5776
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-5776-v2.txt, HDFS-5776-v3.txt, HDFS-5776.txt
>
>
> This is a placeholder of hdfs related stuff backport from
> https://issues.apache.org/jira/browse/HBASE-7509
> The quorum read ability should be helpful especially to optimize read outliers
> we can utilize "dfs.dfsclient.quorum.read.threshold.millis" &
> "dfs.dfsclient.quorum.read.threadpool.size" to enable/disable the hedged read
> ability from client side(e.g. HBase), and by using DFSQuorumReadMetrics, we
> could export the interested metric valus into client system(e.g. HBase's
> regionserver metric).
> The core logic is in pread code path, we decide to goto the original
> fetchBlockByteRange or the new introduced fetchBlockByteRangeSpeculative per
> the above config items.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
