[ https://issues.apache.org/jira/browse/HDFS-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452230#comment-17452230 ]
Janus Chow commented on HDFS-16367: ----------------------------------- [~ferhui] [~hexiaoqiao] [~tasanuma] Could you help to check the ticket and the options? > Out-sync of socket config between client and DataNode > ----------------------------------------------------- > > Key: HDFS-16367 > URL: https://issues.apache.org/jira/browse/HDFS-16367 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Janus Chow > Priority: Major > > We met an issue of timeout while clients writing data to datanodes. The root > cause is the different "dfs.client.socket-timeout" config used by Clients and > DataNode. > Clients updated the config to 3m, DataNodes are using the default config of > 1m. During the stage of "DATA_STREAMING" when writing pipelines, the client > will send heartbeat packets with an interval of half of the timeout, that is > 90seconds, but the socket timeout between the client and the first DataNode > is 60seconds, then the client fails to write the content. > > IMHO, the client config of "dfs.client.socket-timeout" is reasonable to be > configured by clients themselves, thus in a cluster, we should allow the > circumstances of different clients using different client configs. > If so, the problem is the out-sync of some configs between the client and > DataNodes. After checking the code, I think the following 2 options would be > feasible: > # while the client is invoking writeBlock of DataTransferProtocol, we can > transfer the config from client to DataNode, and DataXceiver should process > and update the config. The drawback of this option is we may need to add > arguments to other methods like "readBlock". > # Add a new interface like "syncConfig" to DataTransferProtocol, then after > each time the connection is created, we can invoke the method and do the > config sync up. > Hope to get other options and ideas. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org