[ https://issues.apache.org/jira/browse/HDFS-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved HDFS-3653. ------------------------------- Resolution: Won't Fix > 1.x: Add a retention period for purged edit logs > ------------------------------------------------ > > Key: HDFS-3653 > URL: https://issues.apache.org/jira/browse/HDFS-3653 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode > Affects Versions: 1.1.0 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Priority: Major > > Occasionally we have a bug which causes something to go wrong with edits > files. Even more occasionally the bug is such that the namenode mistakenly > deletes an {{edits}} file without merging it into {{fsimage}} properly -- e.g > if the bug mistakenly writes an OP_INVALID at the top of the log. > In trunk/2.0 we retain many edit log segments going back in time to be more > robust to this kind of error. I'd like to implement something similar (but > much simpler) in 1.x, which would be used only by HDFS developers in > root-causing or repairing from these rare scenarios: the NN should never > directly delete an edit log file. Instead, it should rename the file into > some kind of "trash" directory inside the name dir, and associate it with a > timestamp. Then, periodically a separate thread should scan the trash dirs > and delete any logs older than a configurable time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org