[ https://issues.apache.org/jira/browse/HDFS-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13692225#comment-13692225 ]
Andrew Wang commented on HDFS-4923: ----------------------------------- bq. Either you save the namespace when you stop the namenode or when you start the namenode. The cost of writing multi-GB file cannot be avoided, at least as of now. When HA is configured, the namenode doesn't do a saveNamespace on startup. {code:java} // We shouldn't be calling saveNamespace if we've come up in standby state. MetaRecoveryContext recovery = startOpt.createRecoveryContext(); if (fsImage.recoverTransitionRead(startOpt, this, recovery) && !haEnabled) { fsImage.saveNamespace(this); } {code} > Save namespace when the namenode is stopped > ------------------------------------------- > > Key: HDFS-4923 > URL: https://issues.apache.org/jira/browse/HDFS-4923 > Project: Hadoop HDFS > Issue Type: Improvement > Affects Versions: 3.0.0 > Reporter: Suresh Srinivas > Assignee: Suresh Srinivas > > In rare instances the namenode fails to load editlog due to corruption during > startup. This has more severe impact if editlog segment to be checkpointed > has corruption, as checkpointing fails because the editlog with corruption > cannot be consumed. If an administrator does not notice this and address it > by saving the namespace, recovering the namespace would involve complex file > editing, using previous backups or losing last set of modifications. > The other issue that also happens frequently is, checkpointing fails and has > not happened for a long time, resulting in long editlogs and even corrupt > editlogs. > To handle these issues, when namenode is stopped, we can put it in safemode > and save the namespace, before the process is shutdown. As an added benefit, > the namenode restart would be faster, given there is no editlog to consume. > What do folks think? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira