[ https://issues.apache.org/jira/browse/HADOOP-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12866410#action_12866410 ]
Dmytro Molkov commented on HADOOP-6761: --------------------------------------- Sure Ravi, what is it that you want me to modify? I can add this new parameter to the default conf with description. Would that be enough? Or is there an actual trash documentation that you want me to update? > Improve Trash Emptier > --------------------- > > Key: HADOOP-6761 > URL: https://issues.apache.org/jira/browse/HADOOP-6761 > Project: Hadoop Common > Issue Type: Improvement > Reporter: Dmytro Molkov > Attachments: HADOOP-6761.patch > > > There are two inefficiencies in the Trash functionality right now that have > caused some problems for us. > First if you configured your trash interval to be one day (24 hours) that > means that you store 2 days worth of data eventually. The Current and the > previous timestamp that will not be deleted until the end of the interval. > And another problem is accumulating a lot of data in Trash before the Emptier > wakes up. If there are a couple of million files trashed and the Emptier does > deletion on HDFS the NameNode will freeze until everything is removed. (this > particular problem hopefully will be addressed with HDFS-1143). > My proposal is to have two configuration intervals. One for deleting the > trashed data and another for checkpointing. This way for example for > intervals of one day and one hour we will only store 25 hours of data instead > of 48 right now and the deletions will be happening in smaller chunks every > hour of the day instead of a huge deletion at the end of the day now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.