[
https://issues.apache.org/jira/browse/HDFS-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876743#comment-17876743
]
ASF GitHub Bot commented on HDFS-17556:
---------------------------------------
lfxy commented on PR #6896:
URL: https://github.com/apache/hadoop/pull/6896#issuecomment-2310388780
@zhengchenyu I think it's wrong you described about pendingReconstruction.
The block flow in decommission is:
(1)DatanodeAdminManager::monitor --> (2)neededReconstruction -->
(3)PendingReconstructionBlocks::pendingReconstructions -->
(4)PendingReconstructionBlocks::timedOutItems--> (5)neededReconstruction
If reconstruction task failed in dn, block will flow from (2) to (5)
circularly. When the block is in timedOutItems and we don't check,
DatanodeAdminManager will add block to neededReconstruction repeated.
> Avoid adding block to neededReconstruction repeatedly in decommission
> ---------------------------------------------------------------------
>
> Key: HDFS-17556
> URL: https://issues.apache.org/jira/browse/HDFS-17556
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namanode
> Affects Versions: 3.5.0
> Reporter: caozhiqiang
> Assignee: caozhiqiang
> Priority: Major
> Labels: pull-request-available
>
> In decommission and maintenance process, before added to
> BlockManager::neededReconstruction block will be check if it has been added.
> The check contains if block is in BlockManager::neededReconstruction or in
> PendingReconstructionBlocks::pendingReconstructions as below code.
> But it also need to check if it is in
> PendingReconstructionBlocks::timedOutItems. Or else
> DatanodeAdminDefaultMonitor will add block to
> BlockManager::neededReconstruction repeatedly if block time out in
> PendingReconstructionBlocks::pendingReconstructions.
>
> {code:java}
> if (!blockManager.neededReconstruction.contains(block) &&
> blockManager.pendingReconstruction.getNumReplicas(block) == 0 &&
> blockManager.isPopulatingReplQueues()) {
> // Process these blocks only when active NN is out of safe mode.
> blockManager.neededReconstruction.add(block,
> liveReplicas, num.readOnlyReplicas(),
> num.outOfServiceReplicas(),
> blockManager.getExpectedRedundancyNum(block));
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
