[
https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885083#comment-13885083
]
Liang Xie commented on HDFS-5776:
---------------------------------
bq. we do not check threadpool in enableHedgedReads. This makes it possible
that isHedgedReadsEnabled() returns true while hedged read is actually not
enabled.
i can change to sth like those if you gys want:
{code}
return allowHedgedReads && (HEDGED_READ_THREAD_POOL != null) &&
HEDGED_READ_THREAD_POOL.getMaximumPoolSize() > 0;
{code}
what do you think ?
bq. DFSClient#setThreadsNumForHedgedReads allows users to keep changing the
size of the thread pool.
we definitely need the ability to modify the pool size on the fly, especially
for HBase ops.
bq. Read the thread pool size configuration only when initializing the thread
pool, and the size should be >0 and cannot be changed
Here is the same disagreement, if you guys all still insist on making the pool
size readonly, i can reupload a new patch. Per my few previous operation
experience, it's absolutely inconvenienced to an system ops/admin.
> Support 'hedged' reads in DFSClient
> -----------------------------------
>
> Key: HDFS-5776
> URL: https://issues.apache.org/jira/browse/HDFS-5776
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-5776-v10.txt, HDFS-5776-v2.txt, HDFS-5776-v3.txt,
> HDFS-5776-v4.txt, HDFS-5776-v5.txt, HDFS-5776-v6.txt, HDFS-5776-v7.txt,
> HDFS-5776-v8.txt, HDFS-5776-v9.txt, HDFS-5776.txt
>
>
> This is a placeholder of hdfs related stuff backport from
> https://issues.apache.org/jira/browse/HBASE-7509
> The quorum read ability should be helpful especially to optimize read outliers
> we can utilize "dfs.dfsclient.quorum.read.threshold.millis" &
> "dfs.dfsclient.quorum.read.threadpool.size" to enable/disable the hedged read
> ability from client side(e.g. HBase), and by using DFSQuorumReadMetrics, we
> could export the interested metric valus into client system(e.g. HBase's
> regionserver metric).
> The core logic is in pread code path, we decide to goto the original
> fetchBlockByteRange or the new introduced fetchBlockByteRangeSpeculative per
> the above config items.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
