[ https://issues.apache.org/jira/browse/HDFS-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HDFS-3335: --------------------------------------- Attachment: HDFS-3335.011.patch * increase MAX_OP_SIZE to 100 MB. Since we are limiting a bunch of strings to 1 MB in size, and we can have a few of them in an opcode, it only makes sense. * In the OfflineEditsViewer (oev), we have to initialize the EditLogFileInputStream with startTx = endTxId = INVALID_TXID. This is a special value that indicates that we don't know where the log ends. Previously, I had been passing -1 as the start and end txid, but the convention throughout the code is that only INVALID_TXID may be used for this purpose. This fixes the unit test failure. > check for edit log corruption at the end of the log > --------------------------------------------------- > > Key: HDFS-3335 > URL: https://issues.apache.org/jira/browse/HDFS-3335 > Project: Hadoop HDFS > Issue Type: Improvement > Affects Versions: 0.23.0 > Reporter: Colin Patrick McCabe > Assignee: Colin Patrick McCabe > Attachments: HDFS-3335-b1.001.patch, HDFS-3335-b1.002.patch, > HDFS-3335-b1.003.patch, HDFS-3335-b1.004.patch, HDFS-3335.001.patch, > HDFS-3335.002.patch, HDFS-3335.003.patch, HDFS-3335.004.patch, > HDFS-3335.005.patch, HDFS-3335.006.patch, HDFS-3335.007.patch, > HDFS-3335.008.patch, HDFS-3335.009.patch, HDFS-3335.010.patch, > HDFS-3335.011.patch > > > Even after encountering an OP_INVALID, we should check the end of the edit > log to make sure that it contains no more edits. > This will catch things like rare race conditions or log corruptions that > would otherwise remain undetected. They will got from being silent data loss > scenarios to being cases that we can detect and fix. > Using recovery mode, we can choose to ignore the end of the log if necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira