Kidd5368 opened a new pull request, #4252:
URL: https://github.com/apache/hadoop/pull/4252
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Jira:[HDFS-16566](https://issues.apache.org/jira/browse/HDFS-16566)
Simple case:
RS3-2 ,[0(busy),2,3,4] (1 missing),0 is busy ,
DN0.getNumberOfBlocksToBeReplicated() >= maxReplicationStreams and the priority
of the recovery is not QUEUE_HIGHEST_PRIORITY.
We can get liveblockIndice=[2,3,4], additionalRepl=1.So the DN will get the
LiveBitSet=[2,3,4] and targets.length=1.
According to StripedWriter.initTargetIndices(), 0 will get recovered instead
of 1. So the internal blocks will become [0(busy),2,3,4,0'(excess)].Although NN
will detect, delete the excess replicas and recover the missing block(1)
correctly after the wrong recovery of 0', I don't think this process is
expected and the recovery of 0' is obviously wrong and not necessary.
### How was this patch tested?
RS6-3, [0(busy),1(busy),3,4,5,6,7,8] ,2 is missing.Test the recovery of this
block group to verify there is no excess during the recovery.
### For code changes:
The most important modification is in BlockManager.chooseSourceDatanodes( ).
Add ExcludeReconstructed to save the block in busy DN which should not be
reconstructed when the priority of the recovery is not highest. The changes
also refer to BlockECReconstructionInfo, so the modification of
erasurecoding.proto is necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
