[ https://issues.apache.org/jira/browse/HDFS-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ayush Saxena updated HDFS-15200: -------------------------------- Fix Version/s: 3.2.3 3.1.5 > Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage > ----------------------------------------------------------------------------- > > Key: HDFS-15200 > URL: https://issues.apache.org/jira/browse/HDFS-15200 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Ayush Saxena > Assignee: Ayush Saxena > Priority: Critical > Fix For: 3.3.0, 3.1.5, 3.2.3 > > Attachments: HDFS-15200-01.patch, HDFS-15200-02.patch, > HDFS-15200-03.patch, HDFS-15200-04.patch, HDFS-15200-05.patch > > > Presently {{invalidateBlock(..)}} before adding a replica into invalidates, > checks whether any block replica is on stale storage, if any replica is on > stale storage, it postpones deletion of the replica. > Here : > {code:java} > // Check how many copies we have of the block > if (nr.replicasOnStaleNodes() > 0) { > blockLog.debug("BLOCK* invalidateBlocks: postponing " + > "invalidation of {} on {} because {} replica(s) are located on " + > "nodes with potentially out-of-date block reports", b, dn, > nr.replicasOnStaleNodes()); > postponeBlock(b.getCorrupted()); > return false; > {code} > > In case of corrupt replica, we can skip this logic and delete the corrupt > replica immediately, as a corrupt replica can't get corrected. > One outcome of this behavior presently is namenodes showing different block > states post failover, as: > If a replica is marked corrupt, the Active NN, will mark it as corrupt, and > mark it for deletion and remove it from corruptReplica's and > excessRedundancyMap. > If before the deletion of replica, Failover happens. > The standby Namenode will mark all the storages as stale. > Then will start processing IBR's, Now since the replica's would be on stale > storage, it will skip deletion, and removal from corruptReplica's > Hence both the namenode will show different numbers and different corrupt > replicas. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org