[
https://issues.apache.org/jira/browse/HDFS-16610?focusedWorklogId=776268&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776268
]
ASF GitHub Bot logged work on HDFS-16610:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/May/22 12:01
Start Date: 31/May/22 12:01
Worklog Time Spent: 10m
Work Description: sodonnel opened a new pull request, #4384:
URL: https://github.com/apache/hadoop/pull/4384
### Description of PR
In a cluster with a lot of small files, we encountered a case where fsck was
very slow. I believe it is due to contention with many other threads reading /
writing data on the cluster.
Sometimes fsck does not report any progress for more than 60 seconds and the
client times out. Currently the connect and read timeout are hardcoded to 60
seconds. This change is to make them configurable.
### How was this patch tested?
Tested manually by inserting a sleep into the fsck logic in the NN. I then
adjusted the read timeout to validate I got a timeout or not depending on the
timeout setting.
Issue Time Tracking
-------------------
Worklog Id: (was: 776268)
Remaining Estimate: 0h
Time Spent: 10m
> Make fsck read timeout configurable
> -----------------------------------
>
> Key: HDFS-16610
> URL: https://issues.apache.org/jira/browse/HDFS-16610
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In a cluster with a lot of small files, we encountered a case where fsck was
> very slow. I believe it is due to contention with many other threads reading
> / writing data on the cluster.
> Sometimes fsck does not report any progress for more than 60 seconds and the
> client times out. Currently the connect and read timeout are hardcoded to 60
> seconds. This change is to make them configurable.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
