[ https://issues.apache.org/jira/browse/HDFS-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706455#comment-14706455 ]
Ravikumar commented on HDFS-8892: --------------------------------- Iterating over all replicas is a cause of concern. Is it possible for client to ack a slot made invalid by DN? But my guess is that, it would needlessly complicate this issue. May be the patch itself is applicable for specific use-cases… How about exposing a boolean config "dfs.client.read.shortcircuit.replica.stale.cleanup" and wrapping this small patch in a conditional block? > ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too > ----------------------------------------------------------------- > > Key: HDFS-8892 > URL: https://issues.apache.org/jira/browse/HDFS-8892 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client > Affects Versions: 2.7.1 > Reporter: Ravikumar > Assignee: kanaka kumar avvaru > Priority: Minor > > Currently CacheCleaner thread checks only for cache-expiry times. It would be > nice if it handles an invalid-slot too in an extra-pass of evictable map… > for(ShortCircuitReplica replica:evictable.values()) { > if(!scr.getSlot().isValid()) { > purge(replica); > } > } > //Existing code... > int numDemoted = demoteOldEvictableMmaped(curMs); > int numPurged = 0; > Long evictionTimeNs = Long.valueOf(0); > …. > ….. > Apps like HBase can tweak the expiry/staleness/cache-size params in > DFS-Client, so that ShortCircuitReplica will never be closed except when Slot > is declared invalid. > I assume slot-invalidation will happen during block-invalidation/deletes > {Primarily triggered by compaction/shard-takeover etc..} -- This message was sent by Atlassian JIRA (v6.3.4#6332)