[
https://issues.apache.org/jira/browse/HDFS-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199400#comment-14199400
]
Haohui Mai commented on HDFS-7279:
----------------------------------
I tested the patch in the following setting. Each DN serves 1k concurrent
writers, each of which writes 4GB in total in 64 separate requests.
The old DN frequently ran into GC and timeouts. I no longer observe these
issues after this patch.
> Use netty to implement DatanodeWebHdfsMethods
> ---------------------------------------------
>
> Key: HDFS-7279
> URL: https://issues.apache.org/jira/browse/HDFS-7279
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, webhdfs
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Attachments: HDFS-7279.000.patch, HDFS-7279.001.patch,
> HDFS-7279.002.patch, HDFS-7279.003.patch, HDFS-7279.004.patch,
> HDFS-7279.005.patch, HDFS-7279.006.patch, HDFS-7279.007.patch
>
>
> Currently the DN implements all related webhdfs functionality using jetty. As
> the current jetty version the DN used (jetty 6) lacks of fine-grained buffer
> and connection management, DN often suffers from long latency and OOM when
> its webhdfs component is under sustained heavy load.
> This jira proposes to implement the webhdfs component in DN using netty,
> which can be more efficient and allow more finer-grain controls on webhdfs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
