[ https://issues.apache.org/jira/browse/HDFS-14576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865703#comment-16865703 ]
Dinesh Chitlangia commented on HDFS-14576: ------------------------------------------ Agree with [~jojochuang] that this can still be an issue at 10k node scale. Thanks for filing this jira [~hexiaoqiao] > Avoid block report retry and slow down namenode startup > ------------------------------------------------------- > > Key: HDFS-14576 > URL: https://issues.apache.org/jira/browse/HDFS-14576 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode > Reporter: He Xiaoqiao > Assignee: He Xiaoqiao > Priority: Major > > During namenode startup, the load will be very high since it has to process > every datanodes blockreport one by one. If there are hundreds datanodes block > reports pending process, the issue will be more serious even > #processFirstBlockReport is processed a lot more efficiently than ordinary > block reports. Then some of datanode will retry blockreport and lengthens > restart times. I think we should filter the block report request (via > datanode blockreport retries) which has be processed and return directly then > shorten down restart time. I want to state this proposal may be obvious only > for large cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org