[
https://issues.apache.org/jira/browse/HDFS-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko resolved HDFS-1221.
---------------------------------------
Resolution: Not A Problem
> NameNode unable to start due to stale edits log after a crash
> -------------------------------------------------------------
>
> Key: HDFS-1221
> URL: https://issues.apache.org/jira/browse/HDFS-1221
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.1
> Reporter: Thanh Do
>
> - Summary:
> If a crash happens during FSEditLog.createEditLogFile(), the
> edits log file on disk may be stale. During next reboot, NameNode
> will get an exception when parsing the edits file, because of stale data,
> leading to unsuccessful reboot.
> Note: This is just one example. Since we see that edits log (and fsimage)
> does not have checksum, they are vulnerable to corruption too.
>
> - Details:
> The steps to create new edits log (which we infer from HDFS code) are:
> 1) truncate the file to zero size
> 2) write FSConstants.LAYOUT_VERSION to buffer
> 3) insert the end-of-file marker OP_INVALID to the end of the buffer
> 4) preallocate 1MB of data, and fill the data with 0
> 5) flush the buffer to disk
>
> Note that only in step 1, 4, 5, the data on disk is actually changed.
> Now, suppose a crash happens after step 4, but before step 5.
> In the next reboot, NameNode will fetch this edits log file (which contains
> all 0). The first thing parsed is the LAYOUT_VERSION, which is 0. This is OK,
> because NameNode has code to handle that case.
> (but we expect LAYOUT_VERSION to be -18, don't we).
> Now it parses the operation code, which happens to be 0. Unfortunately, since > 0
> is the value for OP_ADD, the NameNode expects some parameters corresponding
> to that operation. Now NameNode calls readString to read the path, which
> throws
> an exception leading to a failed reboot.
> This bug was found by our Failure Testing Service framework:
> http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html
> For questions, please email us: Thanh Do (than...@cs.wisc.edu) and
> Haryadi Gunawi (hary...@eecs.berkeley.edu)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
