[
https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17754384#comment-17754384
]
ASF GitHub Bot commented on HDFS-17154:
---------------------------------------
Hexiaoqiao commented on code in PR #5941:
URL: https://github.com/apache/hadoop/pull/5941#discussion_r1294147667
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -5950,8 +5952,26 @@ LocatedBlock bumpBlockGenerationStamp(ExtendedBlock
block,
block.setGenerationStamp(nextGenerationStamp(
blockManager.isLegacyBlock(block.getLocalBlock())));
- locatedBlock = BlockManager.newLocatedBlock(
- block, file.getLastBlock(), null, -1);
+ BlockInfo lastBlockInfo = file.getLastBlock();
+ locatedBlock = BlockManager.newLocatedBlock(block, lastBlockInfo,
+ null, -1);
+ if (lastBlockInfo.isStriped() &&
+ ((BlockInfoStriped) lastBlockInfo).getTotalBlockNum() >
+ ((LocatedStripedBlock) locatedBlock).getBlockIndices().length) {
+ // The location info in BlockUnderConstructionFeature may not be
+ // complete after a failover, so we just return all block tokens for a
+ // striped block. This will disrupt the correspondence between
+ // LocatedStripedBlock.blockIndices and LocatedStripedBlock.locs,
+ // which is not used in client side. The correspondence between
+ // LocatedStripedBlock.blockIndices and LocatedBlock.blockToken is
+ // ensured.
+ byte[] indices =
+ new byte[((BlockInfoStriped) lastBlockInfo).getTotalBlockNum()];
+ for (int i = 0; i < indices.length; ++i) {
+ indices[i] = (byte) i;
Review Comment:
Got it. +1.
> EC: Fix bug in updateBlockForPipeline after failover
> ----------------------------------------------------
>
> Key: HDFS-17154
> URL: https://issues.apache.org/jira/browse/HDFS-17154
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Shuyan Zhang
> Assignee: Shuyan Zhang
> Priority: Major
> Labels: pull-request-available
>
> In the method `updateBlockForPipeline`, NameNode uses the
> `BlockUnderConstructionFeature` of a BlockInfo to generate the member
> `blockIndices` of `LocatedStripedBlock`.
> And then, NameNode uses `blockIndices` to generate block tokens for client.
> However, if there is a failover, the location info in
> BlockUnderConstructionFeature may be incomplete, which results in the absence
> of the corresponding block tokens.
> When the client receives these incomplete block tokens, it will throw a NPE
> because `updatedBlks[i]` is null.
> NameNode should just return block tokens for all indices to the client.
> Client can pick whichever it likes to use.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
