[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717260#comment-14717260 ]
Colin Patrick McCabe commented on HDFS-8829: -------------------------------------------- {{DFSClient#dataSocketRecvBufferSize}} seems not to be used anywhere. Since the main performance benefit you saw was with setting the DataNode receive buffer size, how about just creating {{dfs.datanode.transfer.socket.send.buffer.size}} and {{dfs.datanode.transfer.socket.recv.buffer.size}} that are explicitly just for {{DataTransferProtocol}} on the {{DataNode}}? Then we can avoid making any changes to the DFSClient in this patch. "This may effect TCP connection throughput." should be "This may affect TCP connection throughput." > DataNode sets SO_RCVBUF explicitly is disabling tcp auto-tuning > --------------------------------------------------------------- > > Key: HDFS-8829 > URL: https://issues.apache.org/jira/browse/HDFS-8829 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode > Affects Versions: 2.3.0, 2.6.0 > Reporter: He Tianyi > Assignee: He Tianyi > Attachments: HDFS-8829.0001.patch, HDFS-8829.0002.patch, > HDFS-8829.0003.patch > > > {code:java} > private void initDataXceiver(Configuration conf) throws IOException { > // find free port or use privileged port provided > TcpPeerServer tcpPeerServer; > if (secureResources != null) { > tcpPeerServer = new TcpPeerServer(secureResources); > } else { > tcpPeerServer = new TcpPeerServer(dnConf.socketWriteTimeout, > DataNode.getStreamingAddr(conf)); > } > > tcpPeerServer.setReceiveBufferSize(HdfsConstants.DEFAULT_DATA_SOCKET_SIZE); > {code} > The last line sets SO_RCVBUF explicitly, thus disabling tcp auto-tuning on > some system. > Shall we make this behavior configurable? -- This message was sent by Atlassian JIRA (v6.3.4#6332)