[ https://issues.apache.org/jira/browse/HDFS-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303135#comment-14303135 ]
Hudson commented on HDFS-6651: ------------------------------ SUCCESS: Integrated in Hadoop-Yarn-trunk #827 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/827/]) HDFS-6651. Deletion failure can leak inodes permanently. Contributed by Jing Zhao. (wheat9: rev 8cb473124c1cf1c6f68ead7bde06558ebf7ce47e) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotDeletion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeReference.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeSymlink.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/AbstractINodeDiffList.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeWithAdditionalFields.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeMap.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestAclWithSnapshot.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestNestedSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithSnapshotFeature.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectoryWithSnapshotFeature.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java > Deletion failure can leak inodes permanently > -------------------------------------------- > > Key: HDFS-6651 > URL: https://issues.apache.org/jira/browse/HDFS-6651 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Kihwal Lee > Assignee: Jing Zhao > Priority: Critical > Fix For: 2.7.0 > > Attachments: HDFS-6651.000.patch, HDFS-6651.001.patch, > HDFS-6651.002.patch > > > As discussed in HDFS-6618, if a deletion of tree fails in the middle, any > collected inodes and blocks will not be removed from {{INodeMap}} and > {{BlocksMap}}. > Since fsimage is saved by iterating over {{INodeMap}}, the leak will persist > across name node restart. Although blanked out inodes will not have reference > to blocks, blocks will still refer to the inode as {{BlockCollection}}. As > long as it is not null, blocks will live on. The leaked blocks from blanked > out inodes will go away after restart. > Options (when delete fails in the middle) > - Complete the partial delete: edit log the partial delete and remove inodes > and blocks. > - Somehow undo the partial delete. > - Check quota for snapshot diff beforehand for the whole subtree. > - Ignore quota check during delete even if snapshot is present. -- This message was sent by Atlassian JIRA (v6.3.4#6332)