Haiyang Hu created HDFS-17297:
---------------------------------
Summary: The NameNode should remove block from the BlocksMap if
the block is marked as deleted.
Key: HDFS-17297
URL: https://issues.apache.org/jira/browse/HDFS-17297
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu
When call internalReleaseLease method:
{code:java}
boolean internalReleaseLease(
...
int minLocationsNum = 1;
if (lastBlock.isStriped()) {
minLocationsNum = ((BlockInfoStriped) lastBlock).getRealDataBlockNum();
}
if (uc.getNumExpectedLocations() < minLocationsNum &&
lastBlock.getNumBytes() == 0) {
// There is no datanode reported to this block.
// may be client have crashed before writing data to pipeline.
// This blocks doesn't need any recovery.
// We can remove this block and close the file.
pendingFile.removeLastBlock(lastBlock);
finalizeINodeFileUnderConstruction(src, pendingFile,
iip.getLatestSnapshotId(), false);
...
}
{code}
if the condition uc.getNumExpectedLocations() < minLocationsNum &&
lastBlock.getNumBytes() == 0 is met during the execution of UNDER_RECOVERY
logic, the block is removed from the block list in the inode file and marked as
deleted.
However it is not removed from the BlocksMap, it may cause memory leak.
Therefore it is necessary to remove the block from the BlocksMap at this point
as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
