[ https://issues.apache.org/jira/browse/HDFS-8875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HDFS-8875 started by Chris Trezzo. ------------------------------------------ > Optimize the wait time in Balancer for federation scenario > ---------------------------------------------------------- > > Key: HDFS-8875 > URL: https://issues.apache.org/jira/browse/HDFS-8875 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Ming Ma > Assignee: Chris Trezzo > > Balancer has wait time between two consecutive iterations. That is to give > some time for block movement to be fully committed ( return from replaceBlock > doesn't mean the NN's blockmap has been updated and the block has been > invalidated on the source node.). > This wait time could be 23 seconds if {{dfs.heartbeat.interval}} is set to 10 > and {{dfs.namenode.replication.interval}} is to 3. In the case of federation, > given we iterate through all namespaces in each iteration, this wait time > becomes unnecessary as while balancer is processing the next namespace, it > gives the previous namespace it just finished time to commit. > In addition, Balancer calls {{Collections.shuffle(connectors);}} It doesn't > seem necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)