[ https://issues.apache.org/jira/browse/HDFS-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Beaudreault updated HDFS-16261: ------------------------------------- Summary: Configurable grace period around invalidation of replaced blocks (was: Configurable grace period around deletion of invalidated blocks) > Configurable grace period around invalidation of replaced blocks > ---------------------------------------------------------------- > > Key: HDFS-16261 > URL: https://issues.apache.org/jira/browse/HDFS-16261 > Project: Hadoop HDFS > Issue Type: New Feature > Reporter: Bryan Beaudreault > Assignee: Bryan Beaudreault > Priority: Major > > When a block is moved with REPLACE_BLOCK, the new location is recorded in the > NameNode and the NameNode instructs the old host to in invalidate the block > using DNA_INVALIDATE. As it stands today, this invalidation is async but > tends to happen relatively quickly. > I'm working on a feature for HBase which enables efficient healing of > locality through Balancer-style low level block moves (HBASE-26250). One > issue is that HBase tends to keep open long running DFSInputStreams and > moving blocks from under them causes lots of warns in the RegionServer and > increases long tail latencies due to the necessary retries in the DFSClient. > One way I'd like to fix this is to provide a configurable grace period on > async invalidations. This would give the DFSClient enough time to refresh > block locations before hitting any errors. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org