[
https://issues.apache.org/jira/browse/HDFS-17242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792218#comment-17792218
]
ASF GitHub Bot commented on HDFS-17242:
---------------------------------------
xinglin commented on code in PR #6227:
URL: https://github.com/apache/hadoop/pull/6227#discussion_r1412621631
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java:
##########
@@ -564,6 +564,34 @@ private DataStreamer(HdfsFileStatus stat, ExtendedBlock
block,
this.addBlockFlags = flags;
this.maxPipelineRecoveryRetries = conf.getMaxPipelineRecoveryRetries();
this.markSlowNodeAsBadNodeThreshold =
conf.getMarkSlowNodeAsBadNodeThreshold();
+ congestionBackOffMeanTimeInMs = dfsClient.getConfiguration().getInt(
+ HdfsClientConfigKeys.DFS_CLIENT_CONGESTION_BACKOFF_MEAN_TIME,
+ HdfsClientConfigKeys.DFS_CLIENT_CONGESTION_BACKOFF_MEAN_TIME_DEFAULT);
+ congestionBackOffMaxTimeInMs = dfsClient.getConfiguration().getInt(
+ HdfsClientConfigKeys.DFS_CLIENT_CONGESTION_BACKOFF_MAX_TIME,
+ HdfsClientConfigKeys.DFS_CLIENT_CONGESTION_BACKOFF_MAX_TIME_DEFAULT);
+ if (congestionBackOffMeanTimeInMs <= 0 || congestionBackOffMaxTimeInMs <=
0 ||
+ congestionBackOffMaxTimeInMs < congestionBackOffMeanTimeInMs) {
Review Comment:
this looks a bit weird to me. Isn't the following cleaner?
```
if (meanTime <= 0) {
log.WARN("mean time invalid. use default");
meanTime = meanTimeDefault;
}
if (maxTime <= 0) {
log.WARN("max time invalid. use default");
maxTime = maxTimeDefault;
}
if (meanTime > maxTime) {
log.WARN("mean/max time invalid. use default for both");
meanTime = meanTimeDefault;
maxTime = maxTimeDefault;
}
```
> Make congestion backoff time configurable
> -----------------------------------------
>
> Key: HDFS-17242
> URL: https://issues.apache.org/jira/browse/HDFS-17242
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.4.0
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Minor
> Labels: pull-request-available
>
> Currently，if we enable congestion backoff, we will actually invoke
> backOffIfNecessary method. and the backoff time is computed using
> CONGESTION_BACKOFF_MEAN_TIME_IN_MS and CONGESTION_BACK_OFF_MAX_TIME_IN_MS
> which are hard-code. We should better make them configurable to compute
> backoff sleep time flexibly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
