dannytbecker commented on code in PR #6708:
URL: https://github.com/apache/hadoop/pull/6708#discussion_r1558331856
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingDataNodeMessages.java:
##########
@@ -95,16 +95,27 @@ void removeAllMessagesForDatanode(DatanodeDescriptor dn) {
void enqueueReportedBlock(DatanodeStorageInfo storageInfo, Block block,
ReplicaState reportedState) {
+ long genStamp = block.getGenerationStamp();
+ Queue<ReportedBlockInfo> queue = null;
if (BlockIdManager.isStripedBlockID(block.getBlockId())) {
Block blkId = new Block(BlockIdManager.convertToStripedID(block
.getBlockId()));
- getBlockQueue(blkId).add(
- new ReportedBlockInfo(storageInfo, new Block(block), reportedState));
+ queue = getBlockQueue(blkId);
} else {
block = new Block(block);
- getBlockQueue(block).add(
- new ReportedBlockInfo(storageInfo, block, reportedState));
+ queue = getBlockQueue(block);
}
+ // We only want the latest non-future reported block to be queued for each
+ // DataNode. Otherwise, there can be a race condition that causes an old
+ // reported block to be kept in the queue until the SNN switches to ANN and
+ // the old reported block will be processed and marked as corrupt by the
ANN.
+ // See HDFS-17453
+ int size = queue.size();
+ if (queue.removeIf(rbi -> rbi.storageInfo.equals(storageInfo) &&
Review Comment:
I added these null checks, but I needed to remove the reportedState null
check because the null value is used by removeStoredBlock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
