[ https://issues.apache.org/jira/browse/HDFS-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164219#comment-14164219 ]
Ming Ma commented on HDFS-5175: ------------------------------- [~cnauroth], thanks for the suggestion. We want the ability to config different instances of DFSClient with different DSCP values. We ended up using the approach that Lohit mentioned e.g., from application point of view, they will set the configuration property dfs.client.dscp.bit.intvalue and DSClient will call setTrafficClass before establishing the connection. Your suggestion of custom SocketFactory approach will also work. Each instance of DFSClient will create its own instance of DSCPSocketFactory; DSCPSocketFactory also needs to implement Configurable so that it can access the configured DSCP value. If we want to go with custom SocketFactory approach, we can add it to hadoop-common so that applications and other hadoop projects can use it directly. At the higher level, there are different scenarios. The protocol could be based on DFSClient/DataTransferProtocol over TCP, or Hftp/WebHDFS over HTTP. The data transfer should be read or write. The primary use case for us is cross DC remote read scenario with Hftp. In order to config Socket for Hftp scenario, we ended up using httpclient.Protocol.registerProtocol with custom class of ProtocolSocketFactory, given HttpURLConnection doesn't provide a mechanism for custom SocketFactory. As we try to open source our work to support WebHDFS, it brings up the question of "how to config the socket in WebHDFS given httpclient isn't used by HDFS". We did some investigation and found some possible approaches. 1. HttpsURLConnection has setSSLSocketFactory. Need to test out if it works for regular http. 2. Call Socket.setSocketFactroy() for the entire JVM. This requires custom SocketImplFactory to create custom socket based on webHDFS address. We would like to get more inputs from others. > Provide clients a way to set IP header bits on connections > ---------------------------------------------------------- > > Key: HDFS-5175 > URL: https://issues.apache.org/jira/browse/HDFS-5175 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 2.0.5-alpha > Reporter: Lohit Vijayarenu > > It would be very helpful if we had ability for clients to set IP headers when > they make socket connections for data transfers. We were looking into setting > up QoS using DSCP bit and saw that there is no easy way to let clients pass > down a specific value when clients make connection to DataNode. > As a quick fix we did something similar to io.file.buffer.size where client > could pass down DSCP integer value and when DFSClient opens a stream, it > could set the value on socket using setTrafficClass > Opening this JIRA to get more inputs from others who have had experience and > might have already thought about this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)