[
https://issues.apache.org/jira/browse/HDFS-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133268#comment-17133268
]
Ayush Saxena commented on HDFS-15408:
-------------------------------------
Huge in terms of Datanodes? Why will number of datanodes impacting FSCK?
Well if it is number of datanodes, We have tried FSCK with huge loads on about
more than ~10K datanodes and to my knowledge we never faced this issue.
Did you observe this in a production cluster or a test cluster, 1 Min seems
quite big enough. But I suspect this could be some cluster issue. Not against
making it configurable, but...
> Failed execution caused by SocketTimeoutException
> -------------------------------------------------
>
> Key: HDFS-15408
> URL: https://issues.apache.org/jira/browse/HDFS-15408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.1.1
> Reporter: echohlne
> Priority: Major
>
> When I execute command: hdfs fsck /
> in the hadoop cluster to check the health of the cluster, It always report
> an error execution failure like below:
> {code}
> Connecting to namenode via http://hadoop20:50070/fsck?ugi=hdfs&path=%2F
> Exception in thread "main" java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
> at org.apache.hadoop.hdfs.tools.DFSck.doWork(DFSck.java:359)
> at org.apache.hadoop.hdfs.tools.DFSck.access$000(DFSck.java:72)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:159)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:155)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSck.main(DFSck.java:402)
> {code}
> We try to solve this problem by adding a new parameter:
> {color:#de350b}*dfs.fsck.http.timeout.ms*{color} to control the
> connectionTimeout and the readTimeout if the HttpConnection in DFSck.java
> .Please check is it the right way to solve the problem? thanks a lot!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
