[
https://issues.apache.org/jira/browse/HADOOP-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmytro Molkov updated HADOOP-6761:
----------------------------------
Attachment: HADOOP-6761.3.patch
To have backward compatible behaviour the default value of the emptier interval
should be 0, then the trash value interval is used for checkpointing.
> Improve Trash Emptier
> ---------------------
>
> Key: HADOOP-6761
> URL: https://issues.apache.org/jira/browse/HADOOP-6761
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Attachments: HADOOP-6761.2.patch, HADOOP-6761.3.patch,
> HADOOP-6761.patch
>
>
> There are two inefficiencies in the Trash functionality right now that have
> caused some problems for us.
> First if you configured your trash interval to be one day (24 hours) that
> means that you store 2 days worth of data eventually. The Current and the
> previous timestamp that will not be deleted until the end of the interval.
> And another problem is accumulating a lot of data in Trash before the Emptier
> wakes up. If there are a couple of million files trashed and the Emptier does
> deletion on HDFS the NameNode will freeze until everything is removed. (this
> particular problem hopefully will be addressed with HDFS-1143).
> My proposal is to have two configuration intervals. One for deleting the
> trashed data and another for checkpointing. This way for example for
> intervals of one day and one hour we will only store 25 hours of data instead
> of 48 right now and the deletions will be happening in smaller chunks every
> hour of the day instead of a huge deletion at the end of the day now.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
