[ https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17754373#comment-17754373 ]
ASF GitHub Bot commented on HDFS-17154: --------------------------------------- zhangshuyan0 commented on code in PR #5941: URL: https://github.com/apache/hadoop/pull/5941#discussion_r1294132630 ########## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ########## @@ -5950,8 +5952,26 @@ LocatedBlock bumpBlockGenerationStamp(ExtendedBlock block, block.setGenerationStamp(nextGenerationStamp( blockManager.isLegacyBlock(block.getLocalBlock()))); - locatedBlock = BlockManager.newLocatedBlock( - block, file.getLastBlock(), null, -1); + BlockInfo lastBlockInfo = file.getLastBlock(); + locatedBlock = BlockManager.newLocatedBlock(block, lastBlockInfo, + null, -1); + if (lastBlockInfo.isStriped() && + ((BlockInfoStriped) lastBlockInfo).getTotalBlockNum() > + ((LocatedStripedBlock) locatedBlock).getBlockIndices().length) { + // The location info in BlockUnderConstructionFeature may not be + // complete after a failover, so we just return all block tokens for a + // striped block. This will disrupt the correspondence between + // LocatedStripedBlock.blockIndices and LocatedStripedBlock.locs, + // which is not used in client side. The correspondence between + // LocatedStripedBlock.blockIndices and LocatedBlock.blockToken is + // ensured. + byte[] indices = + new byte[((BlockInfoStriped) lastBlockInfo).getTotalBlockNum()]; + for (int i = 0; i < indices.length; ++i) { + indices[i] = (byte) i; Review Comment: The role of `byte[] indices` here is to generate block tokens. In your example, if the client does not need [1,6,7], it will directly skip the corresponding block tokens and not use them. > EC: Fix bug in updateBlockForPipeline after failover > ---------------------------------------------------- > > Key: HDFS-17154 > URL: https://issues.apache.org/jira/browse/HDFS-17154 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Shuyan Zhang > Assignee: Shuyan Zhang > Priority: Major > Labels: pull-request-available > > In the method `updateBlockForPipeline`, NameNode uses the > `BlockUnderConstructionFeature` of a BlockInfo to generate the member > `blockIndices` of `LocatedStripedBlock`. > And then, NameNode uses `blockIndices` to generate block tokens for client. > However, if there is a failover, the location info in > BlockUnderConstructionFeature may be incomplete, which results in the absence > of the corresponding block tokens. > When the client receives these incomplete block tokens, it will throw a NPE > because `updatedBlks[i]` is null. > NameNode should just return block tokens for all indices to the client. > Client can pick whichever it likes to use. > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org