[
https://issues.apache.org/jira/browse/HDFS-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927633#action_12927633
]
Thanh Do commented on HDFS-1479:
--------------------------------
Thanks, Zheng for the explanation.
The reason I couldn't find the AsyncDiskService because I was looking at 0.20.2
where deletion at datanode is done synchronously. Now I find it in 0.21.0.
In general, how do you plan to fix this?
> Massive file deletion causes some timeouts in writers
> -----------------------------------------------------
>
> Key: HDFS-1479
> URL: https://issues.apache.org/jira/browse/HDFS-1479
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 0.20.2
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Priority: Minor
>
> When we do a massive deletion of files, we saw some timeouts in writers who's
> writing to HDFS. This does not happen to all DataNodes, but it's happening
> regularly enough that we would like to fix it.
> {code}
> yyy.xxx.com: 10/10/25 00:55:32 WARN hdfs.DFSClient: DFSOutputStream
> ResponseProcessor exception for block
> blk_-5459995953259765112_37619608java.net.SocketTimeoutException: 69000
> millis timeout while waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.10.10.10:56319
> remote=/10.10.10.10:50010]
> {code}
> This is caused by the default setting of AsyncDiskService, which starts 4
> threads per volume to delete files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
