xuzq created HDFS-14728:
---------------------------

             Summary: RBF:GetDatanodeReport causes a large GC pressure on the 
NameNodes
                 Key: HDFS-14728
                 URL: https://issues.apache.org/jira/browse/HDFS-14728
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: rbf
            Reporter: xuzq


When a cluster contains millions of DNs, *GetDatanodeReport* is pretty 
expensive, and it will cause a large GC pressure on NameNode.
When multiple NSs share the millions DNs by federation and the router listens 
to the NSs, the problem will be more serious.
All the NSs will be GC at the same time.

RBF should cache the datanode report informations and have an option to disable 
the cache.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to