[
https://issues.apache.org/jira/browse/HDFS-16386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell resolved HDFS-16386.
--------------------------------------
Resolution: Fixed
> Reduce DataNode load when FsDatasetAsyncDiskService is working
> --------------------------------------------------------------
>
> Key: HDFS-16386
> URL: https://issues.apache.org/jira/browse/HDFS-16386
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.9.2
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.2.4, 3.3.3
>
> Attachments: monitor.png
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Our DataNode node has 36 disks. When FsDatasetAsyncDiskService is working, it
> will cause a high load on the DataNode.
> Here are some monitoring related to memory:
> !monitor.png!
> Since each disk deletes the block asynchronously, and each thread allows 4
> threads to work,
> This will cause some troubles to the DataNode, such as increased cpu and
> increased memory.
> We should appropriately reduce the number of jobs of the total thread so that
> the DataNode can work better.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
