[ https://issues.apache.org/jira/browse/HDFS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-7990: ------------------------------ Attachment: HDFS-7990.patch IBRs are always sent every heartbeat, even if it only contains delete acks. Note again that the existing behavior is: receiving = next heartbeat, received = immediate, delete = up to 5 mins later. Receiving/delete now have equivalent behavior. Balancer work in HDFS-7967 also discovered that SimulatedFSDataset does not send delete acks - unlike the real dataset - so I fixed that here. > IBR delete ack should not be delayed > ------------------------------------ > > Key: HDFS-7990 > URL: https://issues.apache.org/jira/browse/HDFS-7990 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.0.0-alpha > Reporter: Daryn Sharp > Assignee: Daryn Sharp > Attachments: HDFS-7990.patch > > > HDFS-395 added the incremental BR feature. A concern was avoiding a race > condition with ack-ing block invalidates followed by the directory scanner > re-adding the block before the async delete service removes the block, > possibly resulting in a full BR that includes the previously delete ack-ed > block. > The solution was to batch & delay block deletion acks via a hardcoded > interval of 100 * heartbeat interval (default: 5min). The delay isn't > required: > # The {{FSDatasetSpi}} tracks blocks pending deletions precisely so the > scanner won't re-add the block to the volume map > # Block receiving, received, and deleted go into the same pending report. A > block received event will trigger an immediate IBR which includes the > deletion acks. Ie. The delay is meaningless for all but a quiescent cluster > # Failing to promptly report deleted blocks on a quiescent cluster prevents > the NN from updating the block maps to remove the locations -- This message was sent by Atlassian JIRA (v6.3.4#6332)