[
https://issues.apache.org/jira/browse/HADOOP-18989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
farmmamba updated HADOOP-18989:
-------------------------------
Description:
When we use TestDFSIO tool to test the throughouts of HDFS clusters, we found
it is so slow in the creating controll files stage.
After refering to the source code, we found that method createControlFile try
to create control files serially. It can be improved by using thread pool.
After optimizing, the TestDFSIO tool runs quicker than before.
was:
When we use TestDFSIO tool to test the throughouts of HDFS clusters, we found
it is so slow in the creating controll files stage.
After refering to the source code, we found that method createControlFile try
to create control files serially. It can be improved by using thread pool.
> Use thread pool to improve the speed of creating control files in TestDFSIO
> ---------------------------------------------------------------------------
>
> Key: HADOOP-18989
> URL: https://issues.apache.org/jira/browse/HADOOP-18989
> Project: Hadoop Common
> Issue Type: Improvement
> Components: benchmarks, common
> Affects Versions: 3.3.6
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
>
> When we use TestDFSIO tool to test the throughouts of HDFS clusters, we found
> it is so slow in the creating controll files stage.
> After refering to the source code, we found that method createControlFile try
> to create control files serially. It can be improved by using thread pool.
> After optimizing, the TestDFSIO tool runs quicker than before.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
