[
https://issues.apache.org/jira/browse/HDFS-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058109#comment-14058109
]
Tsz Wo Nicholas Sze commented on HDFS-6651:
-------------------------------------------
Let's number the solutions:
# Complete the partial delete: edit log the partial delete and remove inodes
and blocks.
# Somehow undo the partial delete.
# Check quota for snapshot diff beforehand for the whole subtree.
# Ignore quota check during delete even if snapshot is present.
# do not count snapshot diff into namespace quota
For #1, the resulted namespace tree will depend on quota calculation. This is
problematic if the quota calculation is changed in the future.
Both #2 and #3 give the same user expected behavior. However, these solutions
are very hard to implement.
For #4, it is easy to implement. I think it is fine to relax quota check for
deletion. One concern is that users may be confused -- they may first get
quota exception when creating files. Then, they try deleting some files to
reduce namespace usage. However, the namespace usage not only does not
decrease but increases.
For #5, I like that it can simply the quota calculation. However, the diff
objects do occupy a lot of namespace and changing quota calculation is a kind
of incompatible change.
We should choose either #4 or #5.
> Deletion failure can leak inodes permanently.
> ---------------------------------------------
>
> Key: HDFS-6651
> URL: https://issues.apache.org/jira/browse/HDFS-6651
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Kihwal Lee
> Priority: Critical
>
> As discussed in HDFS-6618, if a deletion of tree fails in the middle, any
> collected inodes and blocks will not be removed from {{INodeMap}} and
> {{BlocksMap}}.
> Since fsimage is saved by iterating over {{INodeMap}}, the leak will persist
> across name node restart. Although blanked out inodes will not have reference
> to blocks, blocks will still refer to the inode as {{BlockCollection}}. As
> long as it is not null, blocks will live on. The leaked blocks from blanked
> out inodes will go away after restart.
> Options (when delete fails in the middle)
> - Complete the partial delete: edit log the partial delete and remove inodes
> and blocks.
> - Somehow undo the partial delete.
> - Check quota for snapshot diff beforehand for the whole subtree.
> - Ignore quota check during delete even if snapshot is present.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
