[
https://issues.apache.org/jira/browse/HDFS-6186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957978#comment-13957978
]
Suresh Srinivas commented on HDFS-6186:
---------------------------------------
Thanks [~atm] for the comments.
One problem with not deleting the blocks for a deleted file is, how does one
restore it? Can we address in this jira pausing deletion after startup and
address the suggestion you have made, along with other changes that might be
necessary, in another jira.
Another jira I was planning to create is, when trash and snapshots are not
enabled, when a large directory (say X number of blocks) is deleted, instead of
deleting the directory, moving it to another directory (say .trash or
/.reserved/.pending_delete). The directory will be deleted after a configured
timeout. What do you think?
> Pause deletion of blocks when the namenode starts up
> ----------------------------------------------------
>
> Key: HDFS-6186
> URL: https://issues.apache.org/jira/browse/HDFS-6186
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Suresh Srinivas
>
> HDFS namenode can delete blocks very quickly, given the deletion happens as a
> parallel operation spread across many datanodes. One of the frequent
> anxieties I see is that a lot of data can be deleted very quickly, when a
> cluster is brought up, especially when one of the storage directories has
> failed and namenode metadata was copied from another storage. Copying wrong
> metadata would results in some of the newer files (if old metadata was
> copied) being deleted along with their blocks.
> HDFS-5986 now captures the number of pending deletion block on namenode webUI
> and JMX. I propose pausing deletion of blocks for a configured period of time
> (default 1 hour?) after namenode comes out of safemode. This will give enough
> time for the administrator to notice large number of pending deletion blocks
> and take corrective action.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
