[
https://issues.apache.org/jira/browse/HDFS-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030574#comment-13030574
]
Todd Lipcon commented on HDFS-1800:
-----------------------------------
bq. Why is the md5 file format "hash\tfile" instead of the format ("hash
*file") produced by md5sum for binary files?
Fixed. I didn't know about the binary marker "*".
bq. MD5FilesUtil should live somewhere like o.a.h.h.util
Good call, fixed.
bq. Per your question in NNStorage#attemptRestoreRemovedStorage I don't think
we should automatically blow away the contents of a failed storage directory.
Not sure which question you're referring to?
bq. Does the version checking removed in NNStorage#getFields get moved
somewhere else?
Yes, I think we have to move it into FSImage where we load the md5 file. This
is the TODO mentioned above about how it probably broke upgrade -- I have
HDFS-1815 filed to track this. The fact that no unit tests are failing due to
this bug means we need to add some real test cases that do upgrade (perhaps
check in a storage dir from a few recent versions)
I think the rest of your comments actually belong on a different JIRA - they
don't seem to reference parts of this patch.
> Extend image checksumming to function with multiple fsimage files
> -----------------------------------------------------------------
>
> Key: HDFS-1800
> URL: https://issues.apache.org/jira/browse/HDFS-1800
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: Edit log branch (HDFS-1073)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: Edit log branch (HDFS-1073)
>
> Attachments: hdfs-1800-prelim.txt, hdfs-1800.txt, hdfs-1800.txt
>
>
> HDFS-903 added the MD5 checksum of the fsimage to the VERSION file in each
> image directory. This allows it to verify that the FSImage didn't get
> corrupted or accidentally replaced on disk.
> With HDFS-1073, there may be multiple fsimage_N files in a storage directory
> corresponding to different checkpoints. So having a single MD5 in the VERSION
> file won't suffice. Instead we need to store an MD5 per image file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
