[
https://issues.apache.org/jira/browse/HDFS-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176234#comment-13176234
]
Eli Collins commented on HDFS-2709:
-----------------------------------
There are two high-level error cases here:
# Error reading edits from the shared mount (eg the NFS mount is flaky/failed)
# Error applying an edit we've read
#1 seems most likely, in this case seems like we should retry a number of times
and then abort (ie the failure isn't transient, in which case we're hosed).
Since we need to apply edits in order I don't think we should readahead and
queue up edits since we're likely to hit errors reading subsequent edits as
well and we can't apply them until the earlier ones have succeeded. Queuing
could potentially give a performance improvement by saving subsequent IO but it
also means the SBN could use a lot more memory than the primary which is bad.
#2 should immediately abort as the edits are checksummed (and same rationale as
ATM's comment above)
> HA: Appropriately handle error conditions in EditLogTailer
> ----------------------------------------------------------
>
> Key: HDFS-2709
> URL: https://issues.apache.org/jira/browse/HDFS-2709
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Todd Lipcon
> Assignee: Aaron T. Myers
> Priority: Critical
>
> Currently if the edit log tailer experiences an error replaying edits in the
> middle of a file, it will go back to retrying from the beginning of the file
> on the next tailing iteration. This is incorrect since many of the edits will
> have already been replayed, and not all edits are idempotent.
> Instead, we either need to (a) support reading from the middle of a finalized
> file (ie skip those edits already applied), or (b) abort the standby if it
> hits an error while tailing. If "a" isn't simple, let's do "b" for now and
> come back to 'a' later since this is a rare circumstance and better to abort
> than be incorrect.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
