Gang Xie created HDFS-13571:
-------------------------------

             Summary: Dead datanode detector
                 Key: HDFS-13571
                 URL: https://issues.apache.org/jira/browse/HDFS-13571
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs-client
    Affects Versions: 3.0.2, 2.6.0, 2.4.0
            Reporter: Gang Xie
             Fix For: 3.0.2


Currently, the information of the dead datanode in DFSInputStream in stored 
locally. So, it could not be shared among the inputstreams of the same 
DFSClient. In our production env, every days, some datanodes dies with 
different causes. At this time, after the first inputstream blocked and detect 
this, it could share this information to others in the same DFSClient, thus, 
the ohter inputstreams are still blocked by the dead node for some time, which 
could cause bad service latency.

To eliminate this impact from dead datanode, we designed a dead datanode 
detector, which detect the dead ones in advance, and share this information 
among all the inputstreams in the same client. This improvement has being 
online for some months and works fine.  So, we decide to port to the 3.0 (the 
version used in our production env is 2.4 and 2.6).

I will do the porting work and upload the code later.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Reply via email to