[ https://issues.apache.org/jira/browse/HDFS-8869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated HDFS-8869: ------------------------------------------ Target Version/s: 2.7.3 (was: 2.7.2) Moving out all non-critical / non-blocker issues that didn't make it out of 2.7.2 into 2.7.3. > Don't mark storages as failed before first block report > ------------------------------------------------------- > > Key: HDFS-8869 > URL: https://issues.apache.org/jira/browse/HDFS-8869 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 2.7.0 > Reporter: Rushabh S Shah > Assignee: Daryn Sharp > > Creating this ticket on behalf of [~daryn]. > Heartbeat processing performs the failed storage check. The DN reports its > storages and any prior missing storages, ex. unique storage id upgrade, are > marked failed. The heartbeat monitor removes all blocks associated to the > failed storage. A replication storm ensues for all blocks on the node. > Eventually the DN block reports for the new storages - up to 15m later for > large clusters. Now the NN has many excess blocks to invalidate. If the > cluster has failed over in the past 24h, ex. rolling upgrade, the standby > gone active will queue the block invalidations which triggers the severe > performance degradation of HDFS-8674 which has been greatly lessened but is > still an issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)