[
https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259173#comment-15259173
]
He Tianyi commented on HDFS-10326:
----------------------------------
[~cmccabe] [~mingma]
https://www.kernel.org/doc/ols/2009/ols2009-pages-169-184.pdf
This document suggests auto tuning is not introduced in Linux 2.4 before 2.4.27
or Linux 2.6 before 2.6.7.
That's very old.
So maybe it's appropriate to enable auto tuning by default.
> Disable setting tcp socket send/receive buffers for write pipelines
> -------------------------------------------------------------------
>
> Key: HDFS-10326
> URL: https://issues.apache.org/jira/browse/HDFS-10326
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode, hdfs
> Affects Versions: 2.6.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> The DataStreamer and the Datanode use a hardcoded
> DEFAULT_DATA_SOCKET_SIZE=128K for the send and receive buffers of a write
> pipeline. Explicitly setting tcp buffer sizes disables tcp stack
> auto-tuning.
> The hardcoded value will saturate a 1Gb with 1ms RTT. 105Mbs at 10ms.
> Paltry 11Mbs over a 100ms long haul. 10Gb networks are underutilized.
> There should either be a configuration to completely disable setting the
> buffers, or the the setReceiveBuffer and setSendBuffer should be removed
> entirely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
