[
https://issues.apache.org/jira/browse/HDFS-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haohui Mai updated HDFS-5270:
-----------------------------
Attachment: TestConcurrentAccess.java
Here is a test to generate concurrent workloads.
I ran the test to write 50000 files into a 3-node cluster using 320 threads.
When running the test multiple times, the test fails (DFSClient retries)
starting from the second runs, while the patch allows the test to run correctly.
> Use thread pools in the datenode daemons
> ----------------------------------------
>
> Key: HDFS-5270
> URL: https://issues.apache.org/jira/browse/HDFS-5270
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Attachments: HDFS-5270.000.patch, TestConcurrentAccess.java
>
>
> The current implementation of the datanode creates a thread when a new
> request comes in. This incurs high overheads for the creation / destruction
> of threads, making the datanode unstable under high concurrent loads.
> This JIRA proposes to use a thread pool to reduce the overheads.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
